Cloudformation template completes deployment before UserData is finished

In the CloudFormation template I am deploying, I am running a few commands in the UserData block. One of these commands starts up a session for NICE DCV: https://aws.amazon.com/hpc/dcv/

It looks as follows:

"UserData": {
    "Fn::Base64" : {
        "Fn::Join" : [
        "",
        [
        dcv create-session --type virtual ",
        " --owner ubuntu",
        " --user ubuntu",
        " my-session, "\n",
        "while ! (dcv list-sessions | grep -q 'my-session'); do sleep 1; done\n"
        ]
        ]
     }
}

First, I create a session with command:

$ dcv create-session --type virtual --owner ubuntu --user ubuntu my-session

Afterwards, I wait to check whether the session has been created successfully with command:

$ while ! (dcv list-sessions | grep -q 'my-session'); do sleep 1; done

The issue I am seeing is that I believe the CloudFormation template is completing it's deployment before the UserData script finishes running. I believe this is the case because if I am quick enough and ssh into the instance, I will see something as follows:

$ dcv list-sessions
There are no sessions available
$ dcv list-sessions
There are no sessions available
$ dcv list-sessions
There are no sessions available
$ dcv list-sessions
There are no sessions available
$ dcv list-sessions
Session: 'my-session' (owner: ubuntu)

which suggests that something was still running in the instance.


How can I make sure that the UserData code is actually respected?

Answers 1

  • To ensure the CloudFormation template waits for the completion of the UserData script, you must do two things:

    1. Add a CreationPolicy to the resource you are targeting (virtual machine in my case).

    2. Add logic in the script to signal its completion. This custom logic uses the cfn-signal utility, which you might have to install in your instance.


    Here's how the template looks now:

    "Properties": {
        "UserData": {
            "Fn::Base64" : {
                "Fn::Join" : [
                "",
                [
                   "curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | sudo python\n",
                   "curl https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz -o /home/ubuntu/aws-cfn-bootstrap.tar.gz\n",
                   "tar -xvzf /home/ubuntu/aws-cfn-bootstrap.tar.gz -C /home/ubuntu/\n",
                   "rm /home/ubuntu/aws-cfn-bootstrap.tar.gz\n",
                   "chmod +x /home/ubuntu/aws-cfn-bootstrap-*/bin/cfn-signal\n",
                   "pip install /home/ubuntu/aws-cfn-bootstrap-*\n",
                   "dcv create-session --type virtual ",
                   " --owner ubuntu",
                   " --user ubuntu",
                   " my-session, "\n",
                   "while ! (dcv list-sessions | grep -q 'my-session'); do sleep 1; done\n"
                   "/home/ubuntu/aws-cfn-bootstrap-*/bin/cfn-signal -e $? ",
                   " --stack ", { "Ref": "AWS::StackName" },
                   " --resource MyInstance" ,
                   " --region ", { "Ref" : "AWS::Region" }, "\n"
                ]
                ]
            }
        }
    },
    "CreationPolicy": {
            "ResourceSignal" : {
                "Count": "1",
                "Timeout": "PT5M"
            }
    }
    

    Breaking the script down.

    This fetches and installs the aws-cfn set of tools, needed for cfn-signal:

    "curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | sudo python\n",
    "curl https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz -o /home/ubuntu/aws-cfn-bootstrap.tar.gz\n",
    "tar -xvzf /home/ubuntu/aws-cfn-bootstrap.tar.gz -C /home/ubuntu/\n",
    "rm /home/ubuntu/aws-cfn-bootstrap.tar.gz\n",
    "chmod +x /home/ubuntu/aws-cfn-bootstrap-*/bin/cfn-signal\n",
    "pip install /home/ubuntu/aws-cfn-bootstrap-*\n",
    

    This is my custom script, which I want to ensure is completed before the cloud formation finishes deployment:

    "dcv create-session --type virtual ",
    " --owner ubuntu",
    " --user ubuntu",
    " my-session, "\n",
    "while ! (dcv list-sessions | grep -q 'my-session'); do sleep 1; done\n"
    

    Finally, I use the cfn-signal utility to signal the termination of the script:

    "/home/ubuntu/aws-cfn-bootstrap-*/bin/cfn-signal -e $? ",
    " --stack ", { "Ref": "AWS::StackName" },
    " --resource MyInstance" ,
    " --region ", { "Ref" : "AWS::Region" }, "\n"
    

    See here for a Windows example.


Related Articles