AWS Systems Manager and New Relic

The AWS Cloud keeps getting easier: Install the New Relic infrastructure agent with AWS Systems Manager. Given the dynamic capabilities of AWS Systems Manager, management and configuration for Amazon Web Services (AWS) cloud infrastructure is more robust than ever before. Observability is a critical part of infrastructure management—and the AWS Systems Manager makes it easier than ever to install, update, or remove the New Relic infrastructure agent when creating Amazon EC2 instances.

This blog post first appeared on the New Relic blog on November 15th, 2019, and was updated on February 17th, 2023.

If you use—or plan to use—New Relic infrastructure monitoring to gather metrics on or help scale your EC2 infrastructure, this post is for you. I’ll show you how easy it is to install supported Linux and Windows infrastructure agents onto your EC2 instances in just a few simple steps.

If you want to jump in and try out installing the New Relic infrastructure agent on just one EC2 instance using our guided installation, you can install the Amazon Linux quickstart.

Inside AWS Systems Manager

To understand how to manage EC2 instances, let’s first look at the overall AWS Systems Management hierarchy. The Systems Manager configures and runs management tasks on managed nodes, like Elastic Compute Cloud (Amazon EC2) instances, edge devices, and on-premises servers or virtual machines. Using Systems Manager, you can:

• Inspect your system’s inventory
• Patch vulnerabilities
• Install additional tools—such as the infrastructure agent, into your infrastructure

For Systems Manager to control a node like an EC2 instance, it requires the use of a Simple Systems Manager (SSM) agent. Since we’re focused on EC2 instances, the SSM agent is the main component and I’ll refer to it as the infrastructure agent for the rest of this post.

Prerequisites

I’m going to assume you already have an EC2 infrastructure. If you haven’t set up Systems Manager yet, follow the AWS documentation for getting set up, and note the Systems Manager prerequisites.

The Command document is the key

With your EC2 infrastructure in place, we need to install the infrastructure agent using the Systems Manager Run Command. The Run Command executes your Command document, a JSON-formatted file containing the parameters, which will populate into a YAML configuration file.

Understanding the individual files

You can use existing Command documents that you can customize for your own needs, but in this case, let’s use a Command document for installing the infrastructure agent that I wrote, called New-Relic_Infrastructure. 

(I’ve already added this document to all available AWS regions—it’s available for anyone to load and use as a starting document, or you could fork it from GitHub.)

You use this Command document to configure parameters in the infrastructure agent’s configuration file (newrelic-infra.yml). As you’ll see in the following steps, Systems Manager reads the New-Relic_Infrastructure Command document and creates a form for you to fill out when you install the agent.

Only two required options must be set: Action and License Key. The default Action is Install, and the value for License Key must be your New Relic ingest license key, which is required to perform any action on the infrastructure agent with the Run Command.

For example, here’s an excerpt of New-Relic_Infrastructure showing two required settings:

{
    "schemaVersion":"2.2",
    "description":"Install, update, or remove the New Relic Infrastructure agent. Infrastructure supports these operating systems: Amazon Linux all versions, CentOS version 6 or higher, Debian version 7 (Wheezy) or higher - Note: SSM not supported on Debian, Red Hat Enterprise Linux (RHEL) version 6 or higher, Ubuntu versions 12.04.*, 14.04.*, and 16.04.* (LTS  versions), and Windows Server 2008, 2012, and 2016 and their service packs. - by Kevin Downs http://linkedin.com/in/kupsand",
    "parameters":
    {
        "action":
        {
            "type":"String",
            "default":"Install",
            "description":"(Required) Install, update, or remove. Note: update only updates the Infrastructure agent, it doesn't update or change the agent configuration.",
            "allowedValues":
            [
                "Install",
                "Update",
                "Remove"
            ]
        },
        "licenseKey":
        {
            "type":"String",
            "description":"(Required) Your New Relic license key.",
            "maxChars":40
        },
}

The rest of the configuration parameters are optional and are documented in the infrastructure agent configuration docs.

Conveniently, this Command document is cross-platform and can install infrastructure agents on supported Linux and Windows instances at the same time.

1. Install the infrastructure agent using the AWS Systems Manager Run Command

Let’s install the infrastructure agent with AWS Systems Manager. (Note: You can execute a Run Command in only one region at a time.)

1. Log in to your AWS console.

2. Navigate to AWS Systems Manager, and select Run Command.

3. Use the document requester drop-down and select Owner, then select Public Documents.

4. Type “New-Relic_Infrastructure” and select Enter. You’ll see the New-Relic_Infrastructure document listed.

After selecting the New-Relic_Infrastructure Command document, you’ll see all the fields available for you to fill out. Required fields are noted with (Required), but any others are optional.

5. Select the Action (in this case, Install) and add your New Relic license key.

The rest of the fields are optional and are used to configure the agent configuration file or add custom attributes. Follow the infrastructure agent configuration documentation to set these as needed. 

The Comment field is a standard SSM field; I recommend entering something that describes the instances you’ve selected to run the Command on, should you need to reference this later.

6. Select the instance or instances on which you want to run the New-Relic_InfrastructureCommand document. Select instances manually, choose a resource group, or specify instance tags.
Note: You can select only instances for the region you are currently working in. You can select multiple instances with different operating systems as long as the infrastructure agent supports those operating systems.

7. Select your output options, and use either Amazon S3 or Enable CloudWatch logs.

8. When you’ve completed all the fields you need, select Run.

2. Review the agent installation output

The next window gives you the option to view the agent installation output by selecting the Instance ID and selecting View output.

From there, you can view the output or error for each step of the Command document. Since the Command document handles the infrastructure agent installation on both Linux and Windows instances, you can see the entries for each. As the following image shows, step one is for Windows, and step two is for Linux.

In this case, the output for Step 2 status was a success and the agent was installed successfully on the Linux EC2 instance.

Let’s take a look at the instance in the New Relic platform. You can see that metrics are starting to roll in.

3. And we’re done here!

Now that the infrastructure agent is installed on the selected instance, and data is streaming to the New Relic console, we’re finished. You’ll see the metrics and data that the infrastructure agent captures and the integration attributes of your EC2 instance. You can use these attributes to filter how you view your infrastructure monitoring and even create saved views to organize your hosts better.

Deploying the New Relic infrastructure monitoring agent doesn’t get much easier than this.

Ready to learn more? Learn about New Relic’s AWS Billing integration to see where you’re spending your budget.