Home › APIs › Export Data Service Sample Scripts

Export Data Service Sample Scripts

(Last updated January 30, 2019)
Below are sample PowerShell and Shell scripts that let you quickly automate the process of exporting data from BEACON. They are intended to show you how to handle interaction with the EDS APIs and do not include error handling.

PowerShell Sample
ps_sample.ps1 calls the EDS v1 Consumption endpoint to get a report of the total consumption along with the number of meters reporting >0 flow in your system.

Shell samples
Range.sh lets you quickly use the BEACON Export Data Service API to get reads over a range of dates.

Flow.sh lets you quickly use the BEACON Export Data Service API to get consumption (flow) data for your meters.

Leak.sh lets you quickly use the BEACON Export Data Service API to get a list of meters with leaks, along with the leak start date and the current leak rate.

Read.sh lets you quickly use the BEACON Export Data Service API to get billing read data for a selection of meters or all of the meters in your utility.

NOTE TO BEACON API USERS IN CANADA
In compliance with Canadian cross-border data regulations, your utility data is or already has been moved to Canada. As a result, if you login to BEACON via beaconama.ca you should adjust all of your API calls to address endpoints at api.beaconama.ca.

For example, change
https://api.beaconama.net/v1/eds/read/
to
https://api.beaconama.ca/v1/eds/read/

This change affects all API endpoint calls for utilities that login to beaconama.ca.

Making API Calls with PowerShell
PowerShell is a task automation scripting language from Microsoft that simplifies making REST API calls and handling JSON and CSV data. Powershell runs on Windows, MacOS and Linux.

Before running Powershell scripts:

Windows

  • Download and install the latest version of PowerShell. (Currently, the latest version is 6.2.4)
  • Add C:\Program Files\PowerShell\6.2.4-1.rhel.7.x86_64.rpm\LK to the PATH environment variable.
  • Open Command Prompt and start PowerShell by typing pwsh.

    • MacOS

    • Download and install the brew package manager from https://brew.sh.
    • Once installed, type brew install –cask powershell into a command-line. This installs PowerShell.

    For more details on using PowerShell, click here.

    Using ps_sample.ps1 to call the EDS API v1 Consumption endpoint

    Copy and paste the script below into a file and name it, for example ps_sample.ps1. This script is run from the command line and requires you to provide four parameters: start_date, end_date, username and password.

    • Enter the desired unit of measure on line 12.
    • If desired, change the resolution to one of quarter_hourly, hourly or monthly.
    • Start COMMAND PROMPT (Windows) or from Terminal (MacOS) type pwsh and press ENTER.
    • Run ps_sample.ps1.
    • When prompted, type a start date and, if desired, time in ISO format on Line 12 after $start_date=“YYYY-MM-DD” or $start_date=“YYYY-MM-DDThh:mm:ssZ” and press ENTER.
    • Type an end date and, if desired, time in ISO format on Line 12 after $end_date=“YYYY-MM-DD” or $end_date=“YYYY-MM-DDThh:mm:ssZ” and press ENTER.
    • Type your username and press ENTER.
    • Type your password and press ENTER to run the script.
    param (
    [Parameter(Mandatory=$true)][string]$start_date,
    [Parameter(Mandatory=$true)][string]$end_date,
    [Parameter(Mandatory=$true)][string]$username,
    [Parameter(Mandatory=$true)][string]$password
)
$encoded_creds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes("$($username):$($password)"))
$basic_auth = "Basic $($encoded_creds)"

$uri = 'https://api.beaconama.net/v1/eds/consumption'
$headers = @{'Authorization'=$basic_auth;'Content-Type'='application/x-www-form-urlencoded'}
$params = @{startDate=$start_date;endDate=$end_date;unit='cubicmeter';outputFormat='CSV';headerColumns='Flow_Time,Flow';resolution='daily'}

$r = Invoke-RestMethod -Uri $uri -Headers $headers -Method POST -Body $params
$status_url = 'https://api.beaconama.net'+$r.statusUrl
While ($true) {
    $r2 = Invoke-RestMethod -Uri $status_url -Headers $headers
    Write-Output $r2
    if ($r2.state -eq 'done') {
        $report_url = 'https://api.beaconama.net' + $r2.reportUrl
        $out_file = "report.csv"
        $result = Invoke-WebRequest -Uri $report_url -Headers $headers -MaximumRedirection 1 -ErrorAction SilentlyContinue -OutFile $out_file
        Break
    } else {
        Start-Sleep -Seconds 3
    }
}
#
#
#

    To run the script named ps_sample.ps1 on Windows type: 

    c:\ pwsh -ExecutionPolicy Bypass -File ps_sample.ps1 -start_date 2017-12-01 -end_date 2017-12-19 -username ZZZ -password YYY

    To run the script named ps_sample.ps1 on MacOS type:

    $ pwsh ./ps_sample.ps1 -start_date 2017-12-01 -end_date 2017-12-19 -username ZZZ -password YYY

    The scripts will create a CSV file called report.csv. After the script has completed its run, you can review the content of this file. It will look like this:

    "Flow_Time","Flow"                                                                                                                                                                                                                 "2017-12-01","6140.00000"                                                                                                                                                                                                           
"2017-12-02","5652.40000"
"2017-12-03","5459.10000"
"2017-12-04","5289.20000"
"2017-12-05","5902.60000"
"2017-12-06","5426.50000"
"2017-12-07","5807.80000"

    In PowerShell, if you are behind a proxy server provide the following parameters: start_date, end_date, your BEACON username and password, and your proxy username and password. Also, pass the following information when making the network request using Invoke-RestMethod or Invoke-WebRequest: 

    -Proxy '<server:port>' -ProxyUseDefaultCredentials '<proxy password>'

    See line 19 in the sample script below.

    To run this version of ps_sample.ps1 behind a proxy server on Windows type: 

    c:\ pwsh -ExecutionPolicy Bypass -File ps_sample.ps1 -proxy '<server:port>' -start_date 2017-12-01 -end_date 2017-12-19 -username ZZZ -password YYY

    To run this version of ps_sample.ps1 behind a proxy server on MacOS type:

    $ pwsh ./ps_sample.ps1 -proxy '<server:port>' -start_date 2017-12-01 -end_date 2017-12-19 -username ZZZ -password YYY

    param (
    [Parameter(Mandatory=$true)][string]$start_date,
    [Parameter(Mandatory=$true)][string]$end_date,
    [Parameter(Mandatory=$true)][string]$username,
    [Parameter(Mandatory=$true)][string]$password,
    [Parameter(Mandatory=$true)][string]$proxy
)

$end_date = Get-Date -UFormat "%Y-%m-%d"
$end_date_for_file_name = Get-Date -UFormat "%Y%m%d-%H%m%S"
$out_file_name = "your-report-$($end_date)-report_$($end_date_for_file_name).csv"
$encoded_creds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes("$($username):$($password)"))
$basic_auth = "Basic $($encoded_creds)"

$uri = 'https://api.beaconama.net/v1/eds/consumption'
$headers = @{'Authorization'=$basic_auth;'Content-Type'='application/x-www-form-urlencoded'}
$params = @{startDate=$start_date;endDate=$end_date;unit='cubicmeter';outputFormat='CSV';headerColumns='Flow_Time,Flow';resolution='daily'}

$r = Invoke-RestMethod -Uri $uri -Proxy $proxy -ProxyUseDefaultCredentials -Headers $headers -Method POST -Body $params
$headers = @{'Authorization'=$basic_auth;'Content-Type'='application/x-www-form-urlencoded'}
$status_url = 'https://api.beaconama.net'+$r.statusUrl

While ($true) {
    $r2 = Invoke-RestMethod -Uri $status_url -Proxy $proxy -ProxyUseDefaultCredentials -Headers $headers
    if ($r2.state -eq 'done') {
        $report_url = 'https://api.beaconama.net' + $r2.reportUrl
        $out_file = "report.csv"
        $result = Invoke-WebRequest -Uri $report_url -Proxy $proxy -ProxyUseDefaultCredentials -Headers $headers -MaximumRedirection 0 -ErrorAction SilentlyContinue
        if ($result.StatusCode -eq 302) {
            Invoke-WebRequest -Uri $result.Headers.Location -OutFile $out_file
            Write-Output "Report created in file: $($out_file)"
        }
        Break
    } else {
        Write-Output $r2.state
        Start-Sleep -Seconds 1
    }

    Making API Calls with Shell Scripts
    Shell scripts are another way to make automated calls to the Web Service APIs. Run these sample scripts on a UNIX-based OS such as Linux or Mac OS X or use Windows with an Open Source toolset such as Cygwin.

    Be sure you are running CURL version 6.58.0 or later. Doing so ensures that the authorization header is not passed during the redirect to fetch the REPORT_URL.

    TIP: When passing export parameters, e.g. Meter_ID=1234, either as query parameters or in the body of an HTTP POST, please be sure to URL encode them.

    Before running any of these Shell scripts:

      Linux and MacOS:

    • Download and install the brew package manager from https://brew.sh.
    • Once installed, type brew install jq into a command-line. This installs a JSON processor called jq.

      • Windows

    • See GitHub for information on installing jq on Windows.

    Using Range.sh
    Get interval meter reads across a range of dates using this API call.

    • Copy and paste the script below into a file and name it Range.sh.
    • Replace username:password on Line 2 with your BEACON user credentials separated by a colon (:). Be sure to include the quote (“) marks. For example, CREDS=”yourname:yourpassword”.
    • Enter a start date and, if desired, time in ISO format on Line 4 after SDATE=“YYYY-MM-DD” or SDATE=“YYYY-MM-DDThh:mm:ssZ”
    • Enter an end date and, if desired, time in ISO format on Line 5 after EDATE=“YYYY-MM-DD” or EDATE=“YYYY-MM-DDThh:mm:ssZ”
    • Enter the desired parameter values on Line 9. For example,
      • after resolution= enter any of daily, hourly or monthly
      • after unit= enter any of acrefeet, ccf, cubicfeet, cubicmeter, gallons or liter
    • Locate the script on your hard drive and type the following into a command-line tool:./Range.sh
    • Press ENTER to run the script. When it completes, copy and paste the results into a new file. Save the file and review it in a spreadsheet or other application that can read CSV files.
    #!/bin/bash
CREDS='username:password'

SDATE='2017-07-31'
EDATE='2017-07-31'
COLS='Account_ID,Meter_ID,Endpoint_SN,Flow,Flow_Unit,Flow_Time'
URL='https://api.beaconama.net/v1/eds/range'

REQ=`curl -s -X POST -u$CREDS -d "resolution=daily&startDate=$SDATE&endDate=$EDATE&unit=cubicmeter&headerColumns=$COLS" $URL`
EDS_UUID=`echo $REQ | jq -r .edsUUID`

# if status returns EDS_UUID, then every 10 seconds checks
# to see if the report is ready. When the export has been completed
# you can fetch the export report. Note the curl option -L is used to
# allow curl to automatically follow redirects.

if [ '$EDS_UUID' != 'null' ]
then
  while true
  do
      sleep 10 # check status every 10 seconds.
      REQ=`curl -s -X GET -u$CREDS https://api.beaconama.net/v1/eds/status/$EDS_UUID`
      echo REQ=$REQ
      REPORT_URL=`echo $REQ | jq -r .reportUrl`
      
      if [[ $REPORT_URL != 'null' ]]
      then
          
          curl -L -s -X GET -u$CREDS https://api.beaconama.net/$REPORT_URL -o $OUTFILE
          
          break
      fi
  done
else
  echo 'ERROR: $REQ'
fi
#
#
#

    Using Flow.sh
    Before running the script:

    • Copy and paste the script below into a file and name it Flow.sh.
    • Replace username:password on Line 2 with your BEACON user credentials separated by a colan (:). Be sure to include the quote marks. For example, CREDS=”yourname:yourpassword”.
    • Enter a start date and, if desired, time in ISO format on Line 4 after SDATE=“YYYY-MM-DD” or SDATE=”YYYY-MM-DDThh:mm:ssZ”
    • Enter an end date and, if desired, time in ISO format on Line 5 after EDATE=“YYYY-MM-DD” or EDATE=“YYYY-MM-DDThh:mm:ssZ”
    • Enter the desired parameter values on Line 9. For example,
      • after resolution= enter any of enter any of daily, hourly or monthly
      • after unit= enter any of acrefeet, ccf, cubicfeet, cubicmeter, gallons or liter
    • Save and name the file Flow.sh.
    • Locate the script on your hard drive, direct your command-line tool to it and type the following into the command-line:./Flow.sh
    • Press ENTER to run the script. When it completes, copy and paste the results into a new file. Save the file and review it in a spreadsheet or other application that can read CSV files.
      • #!/bin/bash
CREDS="username:password"

SDATE="2017-07-31"
EDATE="2017-07-31"
COLS="Account_ID,Meter_ID,Endpoint_SN,Flow,Flow_Unit,Point_1_Read,Point_1_Read_Time,Point_2_Read,Point_2_Read_Time"
URL="https://api.beaconama.net/v1/eds/flow"

REQ=`curl -s -X POST -u$CREDS -d "resolution=daily&startDate=$SDATE&endDate=$EDATE&unit=cubicmeter&headerColumns=$COLS" $URL`
EDS_UUID=`echo $REQ | jq -r .edsUUID`

# if status returns EDS_UUID, then every 10 seconds checks
# to see if the report is ready. When the export has been completed
# you can fetch the export report. Note the curl option -L is used to
# allow curl to automatically follow redirects.

if [ '$EDS_UUID' != 'null' ]
then
  while true
  do
      sleep 10 # check status every 10 seconds.
      REQ=`curl -s -X GET -u$CREDS https://api.beaconama.net/v1/eds/status/$EDS_UUID`
      echo REQ=$REQ
      REPORT_URL=`echo $REQ | jq -r .reportUrl`
      
      if [[ $REPORT_URL != 'null' ]]
      then
          
          curl -L -s -X GET -u$CREDS https://api.beaconama.net/$REPORT_URL -o $OUTFILE
          
          break
      fi
  done
else
  echo 'ERROR: $REQ'
fi
#
#
#

    Using Leak.sh

    Use this sample script to call the /v2/eds/leak endpoint and retrieve a list of meters with leaks, along with the leak start date and the current leak rate.

    • Copy and paste the text below into a text file and name it Leak.sh.
    • Replace “username:password” with your BEACON user credentials.
    • Save the file.
    • Locate the script on your hard drive, direct your command-line tool to it and type the following into the command-line:./Leak.sh
    • Press ENTER to run the script. When it completes, copy and paste the results into a new file. Save the file and review it in a spreadsheet or other application that can read CSV files.
      • #!/bin/bash
CREDS='username:password'

COLS='Account_ID,Meter_ID,Current_Leak_Rate,Current_Leak_Start_Date'
URL='https://api.beaconama.net/v2/eds/leak'

REQ=`curl -s -X POST -u$CREDS -d 'Header_Columns=$COLS' $URL`
EDS_UUID=`echo $REQ | jq -r .edsUUID`

# if status returns EDS_UUID, then every 10 seconds checks
# to see if the report is ready. When the export has been completed
# you can fetch the export report. Note the curl option -L is used to
# allow curl to automatically follow redirects.

if [ '$EDS_UUID' != 'null' ]
then
  while true
  do
      sleep 10 # check status every 10 seconds.
      REQ=`curl -s -X GET -u$CREDS https://api.beaconama.net/v1/eds/status/$EDS_UUID`
      echo REQ=$REQ
      REPORT_URL=`echo $REQ | jq -r .reportUrl`
      
      if [[ $REPORT_URL != 'null' ]]
      then
          
          curl -L -s -X GET -u$CREDS https://api.beaconama.net/$REPORT_URL -o $OUTFILE
          
          break
      fi
  done
else
  echo 'ERROR: $REQ'
fi
#
#
#

      Using Read.sh
      Get billing read data for a selection of meters or all meters in your utility using this sample script.

      • Copy and past this script into a file and name it Read.sh.
      • Replace username:password on Line 2 with your BEACON user credentials separated by a colon (:). Be sure to include the quote marks. For example, CREDS=’yourname:yourpassword’.
      • Enter a start date and, if desired, time in ISO format on Line 4 after SDATE=’YYYY-MM-DD’ or SDATE=’YYYY-MM-DDThh:mm:ssZ’
      • Replace the sample Meter_IDs on Line 6 with real Meter_IDs for any number of meters. Use ampersands (&) to separate Meter_IDs.

        Retrieve reads for all of the meters in your utility by deleting Line 6 of this script.

      • Save and name the file Read.sh.
      • Locate the script on your hard drive, direct your command-line tool to it and type the following into the command-line:./Read.sh
      • Press ENTER to run the script. When it completes, copy and paste the results into a new file. Save the file and review it in a spreadsheet or other application that can read CSV files.
      #!/bin/bash
CREDS='username:password'
DATE='2019-11-07T00:00:00Z'
URL='https://api.beaconama.net/v2/eds/read'
COLUMNS='Location_ID,Service_Point_ID,Meter_ID,Meter_SN,Register_Number,Endpoint_SN,Billing_Read,Read_Unit,Read_Time'
METERS='Meter_ID=11111111&Meter_ID=22222222&Meter_ID=33333333&Meter_ID=44444444'
# ---------------------------------------------------------------------------------------
REQ=`curl -s -u$CREDS -H "Content-Type: application/x-www-form-urlencoded" -d "Date=${DATE}&${METERS}&Header_Columns=${COLUMNS}" -X POST $URL`

# echo ${REQ}

EDS_UUID=`echo $REQ | jq -r .edsUUID`
# if status returns EDS_UUID, then every 10 seconds checks
# to see if the report is ready. When the export has been completed
# you can fetch the export report. Note the curl option -L is used to
# allow curl to automatically follow redirects.
if [ '$EDS_UUID' != 'null' ]
then
  while true
  do
      sleep 10 # check status every 10 seconds.
      REQ=`curl -s -X GET -u$CREDS https://api.beaconama.net/v1/eds/status/$EDS_UUID`
      echo REQ=$REQ
      REPORT_URL=`echo $REQ | jq -r .reportUrl`
      if [[ $REPORT_URL != 'null' ]]
      then
          curl -L -s -X GET -u$CREDS https://api.beaconama.net/$REPORT_URL [-0] $OUTFILE
          break
      fi
  done
else
  echo 'ERROR: $REQ'
fi
#
#
#
Knowledge Base

Meter Reading with ORION® Mobile Read

Create Field Assignments for ORION® Mobile Read

 

Knowledge Base Archive

User Guide
Follow via Email

Enter your email address to get new content notifications by email.