In part four we’ve created an empty AWS lambda application to answer to the Google Smart Home requests, now it’s time to really implement it.
To fullfill the requests coming from the Google Assistant we need to implement 4 REST API in our AWS fullfillment .net Core Web API c# solution:
SYNC
QUERY
EXECUTE
DISCONNECT
REPORT STATE
There is an official library for .net (nuget), but since is bad documented we’ll implement a minimal set of the Google Smart Home SDK, just for the thermostats.
SYNC
The first call from Google is sync, which asks for the list of thermostats. The token for the thermostat backend is in the Authorization header, and the body contains a JSON with the SmartHomeRequest object:
public class SmartHomeRequest
{
public string requestId { get; set; }
public List<RequestInput> inputs { get; set; }
public SmartHomeRequest()
{
inputs = new List<RequestInput>();
}
}
public class RequestInput
{
public string intent { get; set; }
}
With the authorization bearer token in the header (coming from the account linking phase) we can retrieve the list of thermostat, and pack it a SmartHomeResponse object:
With this c# models we can elaborate the answer needed by Google in the POST fullfillment call:
We have given Google the list of thermostats. If we test now in the smartphone Home app, when we do the account pairing with [test] Smart Fullfill we get the list of thermostats:
QUERY
Ask the fullfillment server the current real-time status of one or more devices.
NOTE: there are two ways to know the status of device:
QUERY call to fullfillment server
Report State: the thermostats are constantly connected to the Google Home Graph sending statuses
When the Google Assistant is asked for the status of a thermostat, it issues the fullfillment server a QUERY POST call, asking the current real-time status. We just need to put together a JSON like this with data coming from the IoT backend:
Send one or more commands to the fullfillment server for one or more thermostats.
DISCONNECT
Stop reporting to this user.
Report state
The Home Graph is a database of your house devices, and it will contains our thermostats. To show the temperature of this thermostats, especially in the visual-assistant devices, the database must be kept updated with the latest temperatures coming from the thermostats. Every now and then we should call the report-state-api updating the Home Graph. The best moments to call it are:
After the SYNC fullfillment request
After each QUERY fullfillment request
After every EXECUTE fullfillment request
To call report state API we need a private key. To see the code and how to call the report state API look at the JWT post.
Once the fullfillment API is ready and published on AWS, we insert its public address in the intents tab:
Google will send all its requests to the fullfillment URI.
We will serve this URI with a ASP.NET Core Web API, hosted on an AWS Lambda.
To create a Visual Studio solution that deploys in an AWS Lambda follow my previous post. In this post we will focus on the ASP.NET project. The only difference here, is that we use the ASP.NET Core Web API template:
Delete from the templated project the two sample APIs S3ProxyController and ValuesControlles. Right click on Controllers folder and select Add new: Controller (of type API):
It’s a single command of our voice system. In case of thermostats could be:
Rise temperature
What’s the temperature?
Turn off heating
The Google Assistant will listen to the voice command, translate it into an Intent, and send the intent in JSON format to our fullfillment server (ASP.NET Web API).
Google Smart Home has four type of intents:
SYNC: require the list of devices. It’s the first request done by Google to our fullfillment server. Right after the OAuth exchange. In our case will return the list of thermostats.
QUERY: ask for the current state of the system. It’s a read-only request to know thermostat’s state: temperature, setpoints, power.
EXECUTE: require a command to be executed. For example: rise temperature by 2 degrees.
DISCONNECT: Issued by Google when the user unlink it’s thermostat account from Google Assistant account.
Thermostats
The Google Smart Home ecosystem features a lot of Device Type:
There is no limit on the traits a device can support. For example our thermostat could support also Brightness trait.
TemperatureSetting
This trait defines the most used features of a typical thermostat: temperatures and modes. It’s composed by attibutes, states and commands. These are the ones supported by our thermostat:
Attributes:
availableThermostatModes:
off: turn off thermostat itself, disabling heating/cooling.
heat: set thermostat in heating mode (Winter mode)
cool: set thermostat in cooling mode (Summer mode)
on: turn on thermostat. Restore its last functioning mode.
auto: set thermostat in scheduled mode
States:
thermostatMode: current functioning mode
thermostatTemperatureSetpoint: single target temperature in °C
thermostatTemperatureAmbient: observed temperature in °C
Commands:
ThermostatTemperatureSetpoint
thermostatTemperatureSetpoint: single target temperature to set [float type]
ThermostatSetMode
thermostatMode: set a functioning mode (heat/cool/auto/on/off)
TemperatureRelative
thermostatTemperatureRelativeDegree: rise or lower temperature b a number of degrees (Turn down 5 degrees)
Fullfillment server
Now that we have clear which intents to support, we can start developing our fullfillment server. It basically is:
ASP.NET Core Web API server
A single REST API that accepts JSON input
AWS Lambda to host the Web API server
AWS API Gateway to expose the lambda on internet
The fullfillment server is where the biggest development effort is required, and it’s described in part 4.
Google Smart Home and Alexa skills need to impersonate the user when calling private APIs, so an Account Linking is required.
Like the majority of the API economy, Google and Amazon support the OAuth 2 standard.
I will implement a OAuth server that supports the authorization code flow. This OAuth service will provide 2 endpoints:
Web login page
Token endpoint
It consists of a web login page that authenticates the user in my private API authentication system, and the provide back to Google/Amazon a grant code. With this grant code the Google/Amazon app can obtain from the token endpoint an access token and a refresh token. I will develop it in ASP.NET Core using a serverless AWS Lambda. For small number of calls (under 1 milion/month) it’s almost free. Most AWS services used in this tutorial are paid services, you have been warned!
This is the list of AWS services needed:
IAM: authentication service
CloudFormation
S3 bucket for CloudFormation files
APIGateway to get the Lambda public URL
Prerequisites
We need an AWS account and Visual Studio 2019.
Visual Studio ▶Extensions▶Manage extensions: install AWS Toolkit for Visual Studio 2019 (version now is 1.15.2.1).
The VS solution
Create a new AWS Serverless Application (.NET Core C#):
In the next dialog select the ASP.NET Core Web App blueprint:
We will find a full ASP.NET core project, based on .NET Core 2.1 and a few Razor pages.
The OAuth authorization code flow will redirect the user browser to our login page using such an URL: https://myservice.example.com/auth along with 4 parameters:
client_id
redirect_uri
state
response_type
So let’s add an auth login page that can receive these four parameters.
The login page
The login page is a razor page with a login/password form:
public async Task<IActionResult> OnPostAsync()
{
//TODO: check url params
var login = await _ApiService.LoginAsync(Username, Password);
if (login) {
//TODO: generate a code
string uri = RedirectUri + "?code=XXXXX&state=" + State;
return Redirect(rup);
}
Summary: the login page is just a username/password form that authenticate against my private API backend, create an auth code, and redirect back to the caller, passing the auth code.
Token endpoint
The token endpoint is a Web Api controller:
Create an Api subfolder
Right click and select Add Items▶Api Controller Class
Call it ‘tokenController’
The token endpoint is responsible for two post action:
receive a grant auth code and return refresh+access tokens
receive a refresh token and return an access token
Now that we have a working ASP.NET razor login page, and the token POST endpoint, we need to publish it in an AWS lambda, and it’s where the fun starts!
Visual Studio, with the ASW toolkit, provide a Publish to AWS Lambda… shortcut:
Before publishing we need to understand what Publish to AWS Lambda means. In particular we need to know which AWS services will be created (also to understand what we will pay!).
AWS CloudFormation: think of it as a script that configure and starts all AWS services needed. It’s free.
Lambda function: the serverless function that will run the ASP.NET solution. Free for the first few millions calls.
S3 Bucket: the storage where the cloudformation script and the asp.net zipped solution will be stored. You pay for the disk usage.
API Gateway: the public endpoint that expose the ASP.NET site. Free for the first million calls monthly.
The Visual Studio project is already provided with a CloudFormationserverless.template configuration file that will create automatically all the AWS services required. This collection of services is called a stack.
Prerequisite: Install the AWS CLI to do the advanced administative work. You can test it in PowerShell, enter aws help:
IAM: create the administrative account in AWS to upload and run the lambda
To run the publish wizard Visual Studio needs an AWS user with the proper permissions. Go to AWS console, select IAM service and go to Users▶Add.
Visual Studio will use AWS SDK, so a programmatic access it’s enough:
Step 2: Set Permission. Select the Attach existing policies directly and select
AWSLambdaBasicExecutionRole
AmazonS3FullAccess
AWSCloudFormationFullAccess
Step 3: Tags. Skip this step safely.
Step 4: Review. The review page should look like this:
Click Create user. And be sure to download the CSV file!
This is your only chance to download the CSV file, make sure to backup it safely!
WARNING: The permissions set are not enough. The quickest way to configure the IAM user would be to give all permissions, but this could be a security issue. So we will select only the permissions we really need to create a ASP.NET stack.
Enter the IAM user and add a custom policy:
Add these permissions to the policy:
API Gateway
GET
DELETE
PATCH
POST
Lambda
GetFunction
GetFunctionConfiguration
CreateFunction
DeleteFunction
UpdateFunctionConfiguration
UpdateFunctionCode
AddPermission
RemovePermission
ListTags
TagResources
UntagResources
IAM
ListRolePolicies
ListRoles
ListRoleTags
ListUserTags
GetRole
CreateRole
DeleteRole
PassRole
AttachRolePolicy
DeleteRolePolicy
DetachRolePolicy
PutRolePolicy
Save the policy!
Let’s go back to Visual Studio
The CSV file is all we need to make Visual Studio publish the Lambda. Right click on the ASP.NET project and select Publish to AWS Lambda…:
Click on the ‘+’ icon to upload the CSV:
Enter a Profile name and select the CSV file:
Create a bucket, and give a CloudFormation stack name:
This is the final screen of the configured wizard:
When clicking Publish the wizard will start uploading the lambda and then creating the infrastructure with CloudFormation. At the end the status window will show the progresses:
? Congratulations! Your OAuth server is ready on-air!