Using Grafana to Display Sawtooth Metrics¶
Note
These instructions have been tested on Ubuntu 16.04 only.
This procedure describes how to display Sawtooth metrics with Grafana, using InfluxDB to store the metrics data.
Note
This procedure is for a Ubuntu environment. For a Sawtooth network in Docker containers, there are additional steps to change the configuration of the validator and REST API containers. This guide does not describe these steps.
Prerequisites¶
Docker Engine must be installed
One or more Sawtooth nodes must be configured and in a runnable state
Download the hyperledger/sawtooth-core repository from GitHub with this command:
$ git clone https://github.com/hyperledger/sawtooth-core.git
Set Up InfluxDB¶
InfluxDB is used to store Sawtooth metrics data.
Pull the InfluxDB Docker image from Docker Hub.
$ docker pull influxdb
Create an InfluxDB data directory to provide persistent storage on the local file system.
Note
This step creates the directory
/var/lib/influx-data
. If this path is not appropriate for your host operating system, change the path here and in the next command.$ sudo mkdir /var/lib/influx-data
Start an InfluxDB Docker container. This command exposes port 8086 to the network and uses the persistent storage created in the previous step. It also configures the database name (
metrics
) and creates two users for InfluxDB,admin
andlrdata
.In the following command, replace
{admin-pw}
and{lrdata-pw}
with unique passwords for theadmin
andlrdata
users. Remember to properly escape any special characters in these passwords, such as,@!$
.$ docker run -d -p 8086:8086 -v /var/lib/influx-data:/var/lib/influxdb \ -e INFLUXDB_DB=metrics -e INFLUXDB_HTTP_AUTH_ENABLED=true \ -e INFLUXDB_ADMIN_USER=admin -e INFLUXDB_ADMIN_PASSWORD='{admin-pw}' \ -e INFLUXDB_USER=lrdata -e INFLUXDB_USER_PASSWORD='{lrdata-pw}' \ --name sawtooth-stats-influxdb influxdb
Install and Configure Grafana¶
Build the Grafana Docker image from the Dockerfile that is included in the
sawtooth-core
repository.$ cd sawtooth-core/docker $ docker build . -f grafana/sawtooth-stats-grafana \ -t sawtooth-stats-grafana
Run the Grafana container.
$ docker run -d -p 3000:3000 --name sawtooth-stats-grafana \ sawtooth-stats-grafana
Open the Grafana web page at
http://{host}:3000
.In this URL, replace
{host}
with the IP or Fully Qualified Domain Name (FQDN) of the system running the Grafana Docker container.On the Grafana web page, log in as user
admin
with the passwordadmin
.Change the admin password. First, click on the Grafana spiral icon at the top left of the web page and go to “Admin / Profile”. Next, click on “Change Password”.
Configure Grafana to use InfluxDB as a data source.
- Click on the Grafana spiral icon at the top left of the web page and go to “Data Sources”.
- Click on “Metrics”.
- Change the URL to the host server (IP or FQDN) running the InfluxDB Docker container.
- Under “InfluxDB Details”, set
INFLUXDB_USER
tolrdata
. ForINFLUXDB_USER_PASSWORD
, enter thelrdata
password that was defined when you set up InfluxDB. - Click “Save & Test”.
(Sawtooth 1.0.* releases only) Import the Grafana 1.0 dashboard.
Note
Skip this step for Sawtooth release 1.1 and later, which can use the dashboard that is included in the Grafana Docker container from git master.
- Use one of these methods to get the 1.0 dashboard:
- Find the dashboard in the 1-0 branch at
sawtooth-core/docker/grafana/dashboards/sawtooth_performance.json
- Download the dashboard from GitHub at this location: hyperledger/sawtooth-core/1-0/docker/grafana/dashboards/sawtooth_performance.json
- Find the dashboard in the 1-0 branch at
- Click on the Grafana spiral logo and mouse over “Dashboards”, then click “Import”.
- Click “Upload .json file”.
- Navigate to the location of
sawtooth_performance.json
. - Select “metrics” in the drop-down menu and click “Import”.
- Use one of these methods to get the 1.0 dashboard:
Configure the Sawtooth Validator for Grafana¶
The sawtooth-validator
process reports metrics for the Sawtooth validator.
Use the validator configuration file, /etc/sawtooth/validator.toml
, to
specify the validator settings for Grafana.
If the validator configuration file doesn’t exist yet, copy the template from
/etc/sawtooth/validator.toml.example
to/etc/sawtooth/validator.toml
. For more information, see Validator Configuration File.Note
The default config directory is
/etc/sawtooth/
. For information on finding the config directory in a non-default location, see Path Configuration File.Edit
/etc/sawtooth/validator.toml
. Change the following settings to the values that you defined when you set up InfluxDB:opentsdb_url
: Enter the IP or FQDN:port to the InfluxDB instanceopentsdb_db
: Entermetrics
(the value ofINFLUXDB_DB
)opentsdb_username
: Enterlrdata
(theINFLUXDB_USER
)opentsdb_password
: Enter the password forINFLUXDB_USER_PASSWORD
# The host and port for Open TSDB database used for metrics opentsdb_url = "http://{host}:8086" # The name of the database used for storing metrics opentsdb_db = "metrics" opentsdb_username = "lrdata" opentsdb_password = "{lrdata-pw}"
Note
For
opentsdb_url
, be sure to replace the existing host name with the IP or FQDN of the system running the InfluxDB Docker container.Restart the validator for these changes to take effect.
If the validator was started as a
systemd
service:$ sudo systemctl restart sawtooth-validator
To restart
sawtooth-validator
on the command line, see the appropriate procedure in the Application Developer’s Guide: either Using Ubuntu for Your Development Environment or Ubuntu: Add a Node to the Single-Node Environment.
Configure the Sawtooth REST API for Grafana¶
The sawtooth-rest-api
process reports metrics for the Sawtooth REST API.
Use the REST API configuration file, /etc/sawtooth/rest_api.toml
, to specify
the REST API settings for Grafana.
If the REST API configuration file doesn’t exist yet, copy the template from
/etc/sawtooth/rest_api.toml.example
to/etc/sawtooth/rest_api.toml
. For more information, see REST API Configuration File.Note
The default config directory is
/etc/sawtooth/
. For information on finding the config directory in a non-default location, see Path Configuration File.Modify
opentsdb_url
,opentsdb_db
,opentsdb_username
, andopentsdb_password
to match the values used for the validator.opentsdb_url = "http://{host}:8086" # The name of the database used for storing metrics opentsdb_db = "metrics" opentsdb_username = "lrdata" opentsdb_password = "{lrdata-pw}"
Restart the REST API (
sawtooth-rest-api
) for these changes to take effect.If the REST API was started as a
systemd
service:$ sudo systemctl restart sawtooth-rest-api
To restart
sawtooth-rest-api
on the command line, see the appropriate procedure in the Application Developer’s Guide: either Using Ubuntu for Your Development Environment or Ubuntu: Add a Node to the Single-Node Environment. Ubuntu: Add a Node to the Single-Node Environment.
Configure Telegraf¶
Telegraf runs on the Sawtooth nodes to send operating system and hardware metrics to InfluxDB.
Install Telegraf from the InfluxData repository.
$ curl -sL https://repos.influxdata.com/influxdb.key | sudo apt-key add - $ sudo apt-add-repository "deb https://repos.influxdata.com/ubuntu xenial stable" $ sudo apt-get update $ sudo apt-get install telegraf
Edit
/etc/telegraf/telegraf.conf
to configure Telegraf.$ sudo vi /etc/telegraf/telegraf.conf
Under
[[outputs.influxdb]]
, change the following settings to match the values that you defined when you set up InfluxDB.urls = ["http://{host}:8086"] database = "metrics" username = "lrdata" password = "{lrdata-pw}"
Note
Be sure to replace
{host}
with the IP or FQDN of the system running the InfluxDB Docker container.Restart the Telegraf service.
$ sudo systemctl restart telegraf