[Terraform] aws_iam_policy_attachment, provider produced an unexpected new value: Root object was present 오류 수정하기

 

이번에 새로 진행하는 프로젝트의 인프라는 k8s가 아니라 ECS로 선택했다.

 

BE를 병행하는 입장에서 k8s의 계속해서 바뀌는 업데이트와 복잡성을 따라가기 어렵다 판단해서였다.

 

그래서 차근차근 Terraform으로 ECS를 구축하던 중 IAM을 설정하다 의외의 문제를 발견했다.

 

aws_iam_policy_attachment와 관련된 문제여서 짚고가보려고한다.

 

현상부터 특이하다.

 

1. terraform init (-upgrade)

2. terraform plan를 할때 매번 새로운 upgrade 요소가 나옴

3. terraform apply에서 아래와 같은 에러가 발생

4. 다시 1로 돌아가서 upgade를 해주면 plan에서 새로운 리소스가 잡힘. 2~3의 반복

 

분명 정책을 만들고 attachment를 실행했는데, 매번 새로운 attachment가 생성? 되는건지 새로운 적용요소가 등장했고, 그걸 적용하면 에러가 발생했다.

 

에러 메시지는 다음과 같다.

│ When applying changes to aws_iam_policy_attachment.ecs_task_execution_policy_attachment, provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value: Root object was present,    
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to aws_iam_policy_attachment.fe_execution_policy_attachment, provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value: Root object was present, but now  
│ absent.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

 

전체 에러메시지는 아래와 같다.. 지울 것만 대충 지우고 공유

 

plan > apply > plan > apply 마지막에 에러가 발생한다.

> terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_iam_policy_attachment.ecs_task_execution_policy_attachment will be created
  + resource "aws_iam_policy_attachment" "ecs_task_execution_policy_attachment" {
      + id         = (known after apply)
      + policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
      + roles      = [
          + "ecs-task-execution-role",
        ]
    }

  + resource "aws_iam_policy_attachment" "fe_execution_policy_attachment" {
      + id         = (known after apply)
      + policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
      + roles      = [
          + "fe-ecs-task-execution-role",
        ]
    }

Plan: 2 to add, 0 to change, 0 to destroy.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── 

> terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_iam_policy_attachment.ecs_task_execution_policy_attachment will be created
  + resource "aws_iam_policy_attachment" "ecs_task_execution_policy_attachment" {
      + id         = (known after apply)
      + policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
      + roles      = [
          + "ecs-task-execution-role",
        ]
    }

  # aws_iam_policy_attachment.fe_execution_policy_attachment will be created
  + resource "aws_iam_policy_attachment" "fe_execution_policy_attachment" {
      + id         = (known after apply)
      + policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
      + roles      = [
          + "fe-ecs-task-execution-role",
        ]
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Enter a value: yes

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
> terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_iam_policy_attachment.ecs_task_execution_policy_attachment will be updated in-place
  ~ resource "aws_iam_policy_attachment" "ecs_task_execution_policy_attachment" {
      ~ roles      = [
          - "fe-ecs-task-execution-role",
            # (1 unchanged element hidden)
        ]
        # (3 unchanged attributes hidden)
    }

  # aws_iam_policy_attachment.fe_execution_policy_attachment will be updated in-place
      ~ roles      = [
          - "ecs-task-execution-role",
            # (1 unchanged element hidden)
        ]
        # (3 unchanged attributes hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
> terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_iam_policy_attachment.ecs_task_execution_policy_attachment will be updated in-place
  ~ resource "aws_iam_policy_attachment" "ecs_task_execution_policy_attachment" {
        id         = "ecs-task-execution-policy-attachment"
        name       = "ecs-task-execution-policy-attachment"
      ~ roles      = [
          - "fe-ecs-task-execution-role",
            # (1 unchanged element hidden)
        ]
        # (3 unchanged attributes hidden)
    }

  # aws_iam_policy_attachment.fe_execution_policy_attachment will be updated in-place
  ~ resource "aws_iam_policy_attachment" "fe_execution_policy_attachment" {
        id         = "fe-task-execution-policy-attachment"
        name       = "fe-task-execution-policy-attachment"
      ~ roles      = [
          - "ecs-task-execution-role",
            # (1 unchanged element hidden)
        ]
        # (3 unchanged attributes hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.

Do you want to perform these actions in workspace "testbed"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_iam_policy_attachment.fe_execution_policy_attachment: Modifying... [id=fe-task-execution-policy-attachment]
aws_iam_policy_attachment.ecs_task_execution_policy_attachment: Modifying... [id=ecs-task-execution-policy-attachment]
╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to aws_iam_policy_attachment.fe_execution_policy_attachment, provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value: Root object was present, but now  
│ absent.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to aws_iam_policy_attachment.ecs_task_execution_policy_attachment, provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value: Root object was present,    
│ but now absent.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

 

원인은 공식문서에 딱 박혀있었다.

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy_attachment.html

WARNING:
The aws_iam_policy_attachment resource creates exclusive attachments of IAM policies. Across the entire AWS account, all of the users/roles/groups to which a single policy is attached must be declared by a single aws_iam_policy_attachment resource. This means that even any users/roles/groups that have the attached policy via any other mechanism (including other Terraform resources) will have that attached policy revoked by this resource. Consider aws_iam_role_policy_attachment, aws_iam_user_policy_attachment, or aws_iam_group_policy_attachment instead. These resources do not enforce exclusive attachment of an IAM policy.

 

번역하면 aws_iam_policy_attachment는 모든 사용자/역할/그룹에 단 하나만 연결하라는 거고, 기존에 연결된 부분이 있는데 다시 aws_iam_policy_attachment를 사용하면 기존 연결을 해제해버린다는 내용이다.

 

나는 BE/FE용 컨테이너가 두 종류였기 때문에 별 생각없이 각각의 컨테이너의 IAM 권한에다가 정책을 두번 붙이니까 발생한 일이었다.

resource "aws_iam_policy_attachment" "ecs_task_execution_policy_attachment" {
  name       = "ecs-task-execution-policy-attachment"
  roles      = [aws_iam_role.ecs_task_execution_role.name]
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

resource "aws_iam_policy_attachment" "fe_execution_policy_attachment" {
  name       = "fe-task-execution-policy-attachment"
  roles      = [aws_iam_role.fe_ecs_task_execution_role.name]
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

 

그리고 대안으로 aws_iam_role_policy_attachment, aws_iam_user_policy_attachment, aws_iam_group_policy_attachment를 제안했다.

 

기존리소스 제거

terraform state rm ecs-task-execution-policy-attachment
terraform state rm fe-task-execution-policy-attachment

 

aws_iam_role_policy_attachment. 사용하기

resource "aws_iam_role_policy_attachment" "ecs_task_execution_policy_attachment" {
  role       = aws_iam_role.ecs_task_execution_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

resource "aws_iam_role_policy_attachment" "fe_execution_policy_attachment" {
  role       = aws_iam_role.fe_ecs_task_execution_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

 

이렇게 변경하면 별 무리없이 잘 동작한다.

 

마치며

공식문서를 잘 읽자.