Connect NodeMCU (ESP8266) to AWS IoT

18 min readJul 11, 2024

Whether you’re an enthusiast, a hobbyist, or a developer looking to explore the world of the Internet of Things, welcome to this guide on getting started with NodeMCU and AWS IoT!

We will walk through the process of connecting a NodeMCU, a budget-friendly IoT platform based on the most popular hobby-grade Wi-Fi microcontroller ESP8266, to AWS IoT, a powerful cloud platform by Amazon Web Services.

The tutorial will be relevant not only to the NodeMCU boards but also to any DIY microcontroller that may support the Arduino framework, including Arduino itself!

If you are new either to NodeMCU, Arduino ecosystem, or PlatformIO IDE that we will be using in this guide, you are very welcome to my aged but still up-to-date article: Quick start with NodeMCU v3 (ESP8266), Arduino ecosystem, and PlatformIO IDE

AWS IoT?

Even though I always prefer to keep tutorials as hands-on as possible and avoid spending time on something that can be easily googled, I believe it would be useful to share a few thoughts about how AWS IoT can be relevant to you, what are the benefits and challenges that you may encounter.

AWS IoT is a cloud service for connecting and managing Internet of Things devices provided by Amazon Web Services. Here are the top five reasons why you might want to connect NodeMCU or a similar Wi-Fi-enabled microcontroller to AWS IoT:

Data Storage: AWS IoT allows you to store data from your microcontroller in the cloud, providing a centralized repository for all your IoT data — this makes it easier to manage, analyze, and visualize data from devices.
Data Access: with AWS IoT, you can provide real-time access to data sent from your devices, as well as send data back to the microcontroller — this is crucial for applications that require immediate insights, such as managing smart home devices, tracking assets, or monitoring environmental conditions.
Integrations: AWS IoT integrates seamlessly with other AWS services, such as AWS Lambda for serverless computing, or Amazon S3 for data storage — this allows you to build comprehensive solutions that leverage the full power of the AWS ecosystem.
Computing: microcontrollers, while versatile, have limited processing power and memory. By leveraging AWS IoT, you can perform more intensive computations in the cloud. For example, you can analyze data from multiple sensors, run machine learning models, or process large datasets without overburdening the microcontroller.
Security: AWS IoT offers advanced security features, including encrypted communication, access control, and monitoring — this ensures that your data is secure both in transit and at rest, protecting it from unauthorized access.

While providing a window into a powerful AWS cloud environment, AWS IoT inevitably imposes concerns for hobbyists such as complexity and costs, so let’s unpack this!

Concern: Complexity

AWS IoT is often seen as an enterprise-grade solution, and it also has a steep learning curve, which can be daunting for hobbyists and those working on simple DIY projects. The extensive documentation and complex setup processes might seem overwhelming for beginners or even intermediate users who just want to get their devices connected quickly and efficiently.

But don’t worry! In this guide, I’ll show you how to “cook” AWS IoT at home with your NodeMCU, step-by-step. We’ll break down the process into manageable tasks, ensuring that you can set up a working solution without getting lost in the complexities.

By the end, you’ll have NodeMCU connected to AWS IoT Core, sending and receiving messages seamlessly, all while keeping things as straightforward as possible. No intermediate hardware and software such as adapters, bridges, servers, etc. — just NodeMCU and AWS!

Concern: Costs

AWS IoT Core, like many cloud services, operates on a pay-as-you-go model. This means you’ll only be charged for the resources you actually use, which can be highly cost-effective for small projects and DIY enthusiasts.

AWS IoT Core offers a free tier that includes a generous number of operations per month, making it perfect for getting started without incurring any charges. For most DIY projects, including the one we’ll cover in this tutorial, you’ll be well within the free tier limits.

In case you are beyond your 12 months of free tier, like I am, you will be charged roughly the following (looking into the US East North Virginia region, that I am going to use):

$0.08 per million minutes of connection from your devices to AWS IoT Core;
$1.00 per million messages sent to and from AWS IoT Core.

Let’s say your device is connected to AWS IoT 24/7 and sends and receives a message every 10 seconds, this gives us the following costs per device per month:

60 x 24 x 30 = 43,200 minutes of connection x $0.08 / 1,000,000 = $0.003456
2 x (60 / 10) x 60 x 24 x 30 = 518,400 messages x $1.00 / 1,000,000 = $0.5184

As you can see the costs are quite tolerable, you can get more insights into prices with examples and calculations on their website: AWS IoT Core pricing.

With that, we finally completed the introductory part, so let’s get it done!

Part 1. AWS Infrastructure

So here’s the AWS infrastructure overview of what can be built:

There are several additional AWS components shown on the right, beyond the AWS IoT Core, but they are just an example of how you can further extend the solution, while we are going to focus on the central and left parts:

AWS IoT Core configuration;
Microcontroller (MCU) connection over MQTT to AWS IoT Core;
Device certificate and policy that will grant necessary permissions to establish a connection, publish, and receive messages from AWS IoT Core.

MQTT has gained popularity within the hobbyist community for DIY IoT projects due to its simplicity and efficiency. AWS IoT leverages MQTT as a standard to enable seamless, real-time data exchange between IoT devices and the cloud, making it ideal for connecting constrained devices like the NodeMCU.

Device certificate ensures that the communication between your microcontroller and AWS is encrypted and secure, but also authenticates your device with AWS IoT, confirming its identity — this guarantees that only trusted devices can connect to your AWS infrastructure, maintaining the integrity of your system.

To set up your AWS infrastructure I suggest we use two approaches: using CloudFormation or doing it manually. CloudFormation automates the creation of necessary resources, streamlining the process and saving time. Alternatively, the manual method offers a more hands-on experience, providing deeper insights into each step.

With that said, in our scenario, both paths will take about the same time of less than an hour and will lead to the same results- a fully functional two-way connection between NodeMCU and AWS IoT. Choose the one that suits your preference, and let’s get started!

Step 1.1. Option A: CloudFormation-way

The very first step is to create a device certificate that is still manual because when it’s created we’ll need to download certificates and keys to use in our firmware which is not easily possible with CloudFormation today.

Let’s head over to the “IoT Core” product that you can search for in the header search bar, then select on the left “Manage” -> “Security” -> “Certificates”: https://us-east-1.console.aws.amazon.com/iot/home?region=us-east-1#/certificatehub

Open the “Add certificate” menu and click on the “Create certificate” button. Select the recommended option “Auto-generate new certificate” and switch to the “Active” certificate status.

Click on the “Create” button and download all certificates to your computer.

Screenshot 1.A.3. Download Certificates and Keys

Make sure you have 5 files downloaded and stored in a secure place, for example:

Device certificate: b82df4923d572718eaa95e0ee1e942e17831ee8c743abc70931d8f01d33bbd57-certificate.pem.crt
Public key file: b82df4923d572718eaa95e0ee1e942e17831ee8c743abc70931d8f01d33bbd57-public.pem.key
Private key file: b82df4923d572718eaa95e0ee1e942e17831ee8c743abc70931d8f01d33bbd57-private.pem.key
Root CA certificates: AmazonRootCA1.pem…
…and AmazonRootCA3.pem

Downloaded? Great! Now let’s click on the “Continue” button.

Next, click on the created certificate and make a note of the certificate ID because it will be needed for our next step — CloudFormation deployment.

Now, we are going to utilize CloudFormation to deploy the rest of the required infrastructure components using the template that I prepared for you.

Search for “CloudFormation” in the search bar and go to “Stacks” on the left: https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks

Let’s open the “Create stack” menu on the right and click on the “With new resources (standard)” option.

Keep the “Choose an existing template” option on the top selected and choose “Upload a template file” on the bottom. Then click on the “Choose file” button and upload the CloudFormation template from the GitHub repository: cloudformation.json

After the template file is uploaded to the AWS S3, click on the “Next” button.

Enter “Stack name”, for example, aws-iot-wifi-client, then paste the Certificate ID from the previous step as a first parameter. Keep thing and topic names as default or choose what you like.

Move forward by clicking on the “Next” button. Keep the “Configure stack options” configuration as it is and click again on the “Next” button.

As a last step review the CloudFormation stack details and click on the “Submit” button on the bottom to create the stack. After that, we will be redirected to the list of stacks and the stack we just configured will start the deployment process.

Screenshot 1.A.10. Stack Create in Progress

Wait for a minute or two until the stack is fully deployed which will be indicated by the green CREATE_COMPLETE status. After that, let’s go to the “Outputs” tab and find the connection details that we will be using in the firmware code.

In case the stack fails to deploy, try to repeat the steps paying attention to the parameters. However, this template is dead simple and should not give you any trouble when deploying. Otherwise, if you can see the final status and find output values, everything is fine and you are ready to rock!

One of the benefits of CloudFormation is that you can quickly delete all deployed resources by clicking on the “Delete” button. And don’t forget to manually delete the certificate, since it was created out of the CloudFormation flow!

Step 1.1. Option B: Manual way

For those who are interested, we can also configure all infrastructure manually. Given the number of AWS components, it takes about the same time and concludes with the same results, but CloudFormation-free :)

With this option, we’ll start again by searching for “IoT Core” product in the search bar but choosing “Manage” -> “All Devices” -> “Things” on the left: https://us-east-1.console.aws.amazon.com/iot/home?region=us-east-1#/thinghub

Click on the “Create things” button. Keep the “Create single thing” option selected and move forward by clicking on the “Next” button.

Name the thing (device), for example, aws-iot-wifi-client-thing, and leave the rest of the options untouched, click “Next”.

Screenshot 1.B.3. Specify Thing Properties

In the next step, we keep the recommended option “Auto-generate a new certificate” that will create the same type of certificate as we did with the CloudFormation way.

Screenshot 1.B.4. Configure Device Certificate

Before continuing, we will need to create a policy, click on the “Create policy” button that will open a new window.

Screenshot 1.B.5. Attach Policies to Certificate

First, name the policy, for example, aws-iot-wifi-client-policy. After that, instead of allowing all IoT actions we’ll go with a more secure way and create 4 concrete policy statements:

Allow iot:Connect to allow the thing (device) to connect to AWS IoT;
Allow iot:Subscribe to allow the thing to subscribe to a specific topic filter;
Allow iot:Publish to allow the thing to publish messages on a specific topic;
Allow iot:Receive to allow the thing to receive messages from a specific topic.

We’ll use the same topic to subscribe, publish to, and receive messages from, so the resource name for the last 3 statements should be the same, for example aws-iot-wifi-client-topic. There is a difference however between operations of subscribing, publishing, and receiving messages: the first should grant permissions for topicfilter/aws-iot-wifi-client-topic instead of topic/aws-iot-wifi-client-topic.

Resources ARNs for the policy are built according to the following format:

arn:aws:iot:${AWS::Region}:${AWS::AccountId}:client/${Thing}

Where ${AWS::Region} is the region where you create the resources, us-east-1 in my case, and ${AWS::AccountId} is your numeric account ID, which you can obtain by clicking on your account name in the header on the right. The last part is the resource that is different per statement.

To make things easier, click on the “JSON” button on the right in the “Policy document” section and copy and paste the following JSON making sure the region and account ID are corrected to yours:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iot:Connect",
      "Resource": "arn:aws:iot:us-east-1:123456789012:client/aws-iot-wifi-client-thing"
    },
    {
      "Effect": "Allow",
      "Action": "iot:Subscribe",
      "Resource": "arn:aws:iot:us-east-1:123456789012:topicfilter/aws-iot-wifi-client-topic"
    },
    {
      "Effect": "Allow",
      "Action": "iot:Publish",
      "Resource": "arn:aws:iot:us-east-1:123456789012:topic/aws-iot-wifi-client-topic"
    },
    {
      "Effect": "Allow",
      "Action": "iot:Receive",
      "Resource": "arn:aws:iot:us-east-1:123456789012:topic/aws-iot-wifi-client-topic"
    }
  ]
}

Review the policy settings and click on the “Create” button.

After the policy is created, switch to the previous window, if needed, refresh the view, and select the policy, then click on the “Create thing” button.

Screenshot 1.B.9. Attach Policies to Certificate

Right after the device is created, you will be presented with a popup where you need to download all certificates and keys to your computer — the same way as if you also did Option A.

Screenshot 1.B.10. Download Certificates and Keys

Make sure you have 5 files downloaded and stored in a secure place, for example:

Device certificate: b82df4923d572718eaa95e0ee1e942e17831ee8c743abc70931d8f01d33bbd57-certificate.pem.crt
Public key file: b82df4923d572718eaa95e0ee1e942e17831ee8c743abc70931d8f01d33bbd57-public.pem.key
Private key file: b82df4923d572718eaa95e0ee1e942e17831ee8c743abc70931d8f01d33bbd57-private.pem.key
Root CA certificates: AmazonRootCA1.pem…
…and AmazonRootCA3.pem

If everything is stored, click on the “Done” button.

Hooray! We just created all the required infrastructure in AWS! Congratulations!

Step 1.2. Device Data Endpoint

Last, but not least step in AWS for now, is to find the device data endpoint that will be needed to configure the firmware. Let’s go from the “IoT Core” product to the “Settings” section in the left menu.

Make a note of the device endpoint and that’s it, let’s move to the firmware part!

Part 2. NodeMCU Firmware

We will use PlatformIO IDE and Arduino framework to write code for the firmware. If you are completely new in this environment, a good idea would be to start with my other article that will get you up and running: Quick start with NodeMCU v3 (ESP8266), Arduino ecosystem, and PlatformIO IDE

You can create a completely new project with PlatformIO configuring NodeMCU as board and using Arduino framework, or download the GitHub repository of a simple library that I created to provide an abstraction to connect Wi-Fi microcontrollers to AWS IoT: AwsIotWiFiClient — it already has the code we are going to write in src/main.cpp, but still can serve as a convenient scaffolding for this project.

Step 2.1. Connect to Wi-Fi

First things first, we need to connect NodeMCU to your local Wi-Fi network.

Let’s go to the “Libraries” page of the PlatformIO extension, where you can search for “WiFiManager” — we are interested in the one created by “tzapu”.

Navigate to the “Installation” tab, and copy tzapu/WiFiManager @ ^0.16.0 to the platformio.ini file so PlatformIO will download and install this library for us.

Screenshot 3. Configuring Platformio Project

What we want right now is to provide NodeMCU with an SSID and password that it can use to connect to a Wi-Fi network. Copy and paste the following code to src/main.cpp:

#include "Arduino.h"

#include "ESP8266WiFi.h"

#include "DNSServer.h"
#include "ESP8266WebServer.h"
#include "WiFiManager.h"

void setup()
{
  Serial.begin(9600);
  WiFiManager wifiManager;
  wifiManager.autoConnect("AwsIotWiFiClient");
  Serial.println("Connected!");
}

void loop()
{
  Serial.println("Idle...");
  delay(1000);
}

Connect NodeMCU to your computer, flash the firmware, and open the serial monitor (terminal) in PlatformIO.

Next, find the access point (AwsIotWiFiClient) created by NodeMCU on your computer or mobile phone in Wi-Fi settings and configure access to your local Wi-Fi network by providing SSID and password through the captive portal that will be opened shortly after you connect your device to the access point — similar to how it’s done in my quick-start article.

Screenshot 4. Establishing Wi-Fi Connection

If everything is fine, NodeMCU will print Connected! to the terminal from the setup() function and start writing Idle… from the loop() function. Now NodeMCU is connected to your local Wi-Fi network, brilliant!

Step 2.2. Configure Secrets

As a next step, we are going to update src/Secrets.h file with credentials to allow NodeMCU to connect to AWS IoT Core. If you started a new project, you can find this file in the library GitHub repository: Secrets.h

Paste the device data endpoint from step 1.2 in the endpoint constant;
Client ID in the clientId constant, for example, aws-iot-wifi-client-thing;
The topic name that the device will send messages to in the publishTopicName constant, for example, aws-iot-wifi-client-topic;
The topic name that the device will be subscribed to in the subscribeTopicFilter constant, in our scenario should be the same as in #3.

After that, we are going to copy certificates and key contents into the corresponding constants:

Device certificate from the b82df4923d572718eaa95e0ee1e942e17831ee8c743abc70931d8f01d33bbd57-certificate.pem.crt file;
Private key file from the b82df4923d572718eaa95e0ee1e942e17831ee8c743abc70931d8f01d33bbd57-private.pem.key file;
Root CA certificate from the AmazonRootCA1.pem file.

And with that done, our Secrets.h is complete!

Step 2.3. Set up AWS IoT Wi-FI Client

Our next step is to configure the library I created for the task we cover: AwsIotWiFiClient (link to PlatformIO Registry).

This library consists of one class that encapsulates setting up the most popular client libraries for MQTT messaging — PubSubClient, and secure SSL/TLS communication for ESP8266 and ESP32 — WiFiClientSecure.

AwsIotWiFiClient library in its turn, helps to connect to AWS IoT and establish two-way communication, allowing you to focus on implementing business logic, such as reading sensors, exchanging data, switching pins, and blinking LEDs, of course!

If earlier you used the GitHub repository as a starter for your project you already have everything needed to move further. Otherwise, you can install this library the same way as WiFiManager, by searching for “AwsIotWiFiClient” in the “Libraries” page of the PlatformIO extension and adding loginov-rocks/AwsIotWiFiClient @ ^0.1.0 to the platformio.ini file.

Let’s switch over to the src/main.cpp and add a few headers and global variables required by the library to operate:

#include "AwsIotWiFiClient.h"
#include "Secrets.h"

AwsIotWiFiClient awsIotWiFiClient;  

BearSSL::X509List trustAnchorCertificate(rootCaCertificate);
BearSSL::X509List clientCertificate(deviceCertificate);
BearSSL::PrivateKey clientPrivateKey(privateKeyFile);

Next, we are going to configure the awsIotWiFiClient object:

Set certificates to establish secure communication;
Set device data endpoint;
Set client ID;
Set topic name to subscribe to incoming messages;
Define a callback that will be triggered when incoming messages are received.

Define a separate function setupAwsIotWiFiClient() that will configure the awsIotWiFiClient object for us:

void setupAwsIotWiFiClient()
{
  Serial.println("Setting up AWS IoT Wi-Fi Client...");

  awsIotWiFiClient.setDebugOutput(true)
      .setCertificates(&trustAnchorCertificate, &clientCertificate, &clientPrivateKey)
      .setEndpoint(endpoint)
      .setReceiveMessageCallback(receiveMessage)
      .setClientId(clientId)
      .setSubscribeTopicFilter(subscribeTopicFilter)
      .connect();

  Serial.println("AWS IoT Wi-Fi Client setup was successful!");
}

All variables except certificates and private key will be picked up from the src/Secrets.h header file. What’s missing is the definition of the recieveMessage() callback function, let’s add a placeholder at the root level as well:

void receiveMessage(char *topic, byte *payload, unsigned int length)
{
  // TODO
}

Now, we will call the setupAwsIotWiFiClient() function from the setup() function, so add the following:

void setup()
{
  // ...
  setupAwsIotWiFiClient();
  Serial.println("Setup was successful!");
}

The last step is to add awsIotWiFiClient.loop() into the loop() function so the library can do its magic in runtime:

void loop()
{
  awsIotWiFiClient.loop();
}

Let’s flash the firmware to NodeMCU and open the serial monitor.

If the device can’t connect to the previously configured Wi-Fi network, repeat the steps to configure SSID and password through the captive portal. If everything is fine you should see Setup was successful! message in the terminal.

Screenshot 6. Setting up AWS IoT Wi-Fi Client

Lastly, we can verify whether NodeMCU is actually connected to AWS IoT: in the AWS console go to “IoT Core”, and then click on the “Monitor” on the left and switch to the “IoT Metrics” tab — it should show some statistics for Connect and Ping events similar to the screenshot below:

Amazing! AWS IoT Wi-FI Client setup is successful and our NodeMCU just connected to the AWS IoT!

Step 2.4. Publish Messages

A few things left to complete our journey: let’s try to publish messages from NodeMCU to AWS IoT. For that, we need to have the topic name and a message and call the following method of the awsIotWiFiClient object:

awsIotWiFiClient.publishMessage(publishTopicName, "Hello, world!");

As simple as that! But let’s make it a bit more useful by publishing messages sent from your computer over the terminal to NodeMCU.

The approach could be to read characters coming from the serial port until getting some predefined termination character, for example ;, and then publishing it as a message to AWS IoT:

String inputMessage = "";

// ...

void loop()
{
  while (Serial.available() > 0)
  {
    char input = Serial.read();

    if (input == ';')
    {
      Serial.println();
      Serial.println("Publishing message: " + inputMessage);
      const char *charArray = inputMessage.c_str();
      awsIotWiFiClient.publishMessage(publishTopicName, charArray);
      Serial.println("Message published!");
      inputMessage = "";
    }
    else
    {
      Serial.print(input);
      inputMessage += input;
    }
  }

  awsIotWiFiClient.loop();
}

We start by adding the inputMessage variable to the global scope, and then in the loop() function we read the serial port by one character at a time when incoming data is available, comparing it with the predefined termination character.

In case it’s ;, we convert the string inputMessage serving as a buffer for incoming serial characters into the characters array and publish it to AWS IoT. Otherwise, we keep adding the input characters to the inputMessage string.

Let’s flash the NodeMCU firmware with updated code and test if we can receive any messages in the AWS Console. For that head over to the “IoT Core” page, then “Test” -> “MQTT test client” in the left menu, enter the topic name, for example, aws-iot-wifi-client-topic, and click on the “Subscribe” button to establish a connection between your opened AWS console and the IoT Core server.

Now, connect over the terminal to NodeMCU and try to input some messages ending with ;

Screenshot 8. Publishing Message from Terminal

Switch over to the MQTT test client and check received messages.

Screenshot 9. Receiving Message in AWS Console

Voilà!

Step 2.5. Receive Messages

Now, to the last part: let’s fill in the receiveMessage() function placeholder that will be triggered whenever a message arrives on the topic that we subscribed to:

void receiveMessage(char *topic, byte *payload, unsigned int length)
{
  Serial.print("Message received: ");
  Serial.write(payload, length);
  Serial.println();
}

It’s important to keep the function signature as defined because it is used by the awsIotWiFiClient object and must conform to the defined signature of the callback. What happens inside it is completely up to you!

With that done, let’s do our very last test: flash the firmware, connect with the serial monitor, and make sure the MQTT test client is up and running as well. Type any message into the “Message payload” textbox and click on the “Publish” button — it will immediately be shown below since the test client also listens to the same topic.

Screenshot 10. Publishing Message from AWS Console

Now back to the terminal to see if the message was received…

Screenshot 11. Receiving Message in Terminal

And it was, wonderful!

Conclusion

Good work! We successfully learned how to connect NodeMCU to AWS IoT, opening up endless possibilities for your IoT projects. With the power of AWS IoT and the flexibility of the NodeMCU, you can now securely transmit data and create applications that interact with the AWS cloud!

Experiment with buttons, sensors, and actuators, gather and analyze data in real-time, and automate processes using AWS services. The AwsIotWiFiClient library makes it easier to integrate your NodeMCU or any other Wi-Fi microcontroller with AWS IoT, so dive in and explore its features to tailor your IoT solutions to your specific needs.

Happy tinkering, and enjoy the journey of building connected DIY projects!

That’s all Folks!