Skip to content

Commit 4bb5094

Browse files
danielpatenaudedanielpatenaude
committed
Updated scripts to include Google Sheet functionality
1 parent dbcff65 commit 4bb5094

5 files changed

Lines changed: 324 additions & 74 deletions

File tree

‎README.md‎

Lines changed: 48 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,67 @@
11
# python_enlighten_api
2-
Enphase Enlighten API application to pull data and monitor panel performance and send alerts on anomalies
2+
Enphase Enlighten API & Google Sheets application to pull data and monitor panel performance and populate a Google Sheet with historical and visual data. This allows tracking individual panel performance over the lifetime of the system. Most of the functionlaity provided here is also provided by the Enphase Enlighten website and app, but this allows for granular panel tracking and performance over time.
3+
4+
<p align="center">
5+
<img src="solar_performance_example.png" width="500">
6+
</p>
37

48
## Requirements
59
Requires Python 3.6.8 or later installed. Much effort has been taken to ensure this application does not require additional modules besides what is included standard with Python.
610

11+
pip install requests
12+
pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
13+
714
## Getting Started
815

16+
### Script/API Setup
917
1. Allow app access to your Enlighten Account:
1018
* Follow the instructions here to add a new application to your developer account: https://developer.enphase.com/docs/quickstart.html
1119
2. Open run_inverter_daily_stats.py and set the following in the User Settings section:
1220
* user_id: within MyEnlighten go to your account Settings and find the API Settings user ID
1321
* site_id: within MyEnlighten the Site ID should be displayed (or if using the web browser should be in the URL [https://enlighten.enphaseenergy.com/pv/systems/:SITE_ID/overview]
14-
3. Set the following in the Enlighten Settings section:
22+
* api_url: The Enphase API, most likely: https://api.enphaseenergy.com/api/v2/systems
1523
* api_key: within the https://developer.enphase.com/admin/applications page copy your app's API Key
24+
* spreadsheet_id: The SpreadSheet ID from Google Sheets. See step 5.
25+
4. Setup a Google API key for your python script and put the following files in your main working script directory: credentials.json, token.pickle.
26+
* https://developers.google.com/sheets/api/quickstart/python
27+
* Allow your API token to access your Google Sheets account: https://console.developers.google.com/apis/api/sheets.googleapis.com/overview?
28+
5. Google Sheet Setup
29+
* The google sheet can be accessed from: https://docs.google.com/spreadsheets/d/1B9pT3fS1Vb2RyltW2USYqwb0Y48A22HzXnojsDLgvhw/edit?usp=sharing. It is a working copy of an example system setup. You may need to manually clear and remove data to use.
30+
How to Setup:
31+
1. You'll need to make a copy of this sheet to your personal Google Drive.
32+
2. Populate each of your inverter serial numbers into the 'Panel Data-Template' Sheet.
33+
3. Copy and Paste, using values and transpose, your inverter serial numbers into the 'Last 7 Days' sheet.
34+
4. Update the 'Dashboard' Sheet panel layout to match your panels.
35+
5. Update the 'Dashboard' Sheet panel numbers and serial numbers to match your panel data
36+
* Note: Enphase Enlighten did not provide a good way to do this. So I manually had to match up each inverter serial number with the panel number in the layout by tracking panel energy produced over a few days on the 'Panel Data-<Year>' sheet vs the Enphase Enlighten website/app. After a few days each panel's historical data output allowed me to match up each panel on the Dashboard/Panel Data Sheet with the layout of the Enphase app.
37+
38+
### Running
39+
The Enlighten API has a long lag time between when data is updated on their end. If you run these scripts once a day after the Enlighten data updates AND before your solar is producing power (e.g.: 4am) you get the total lifetime power produced by each inverter, including the previous day.
40+
41+
Run with run_inverter_daily_stats.sh or copy the logic this script is using.
42+
43+
#### Setting Automated Cron Jobs
44+
If you're using Linux, you can add these scripts to crontab jobs to run automatically at night by:
45+
46+
Run:
47+
48+
>crontab -e
49+
Add a crontab job:
50+
51+
# At 4am local time run the python script via shell script to ensure we're in the right directory
52+
0 4 * * * /home/<user>/python_enlighten_api/run_inverter_daily_stats.sh >> /home/<user>/python_enlighten_api/cron.log 2>&1
1653

1754
## Enphase Enlighten API Documentation
1855

1956
* https://developer.enphase.com/docs
2057

21-
## Scripts
58+
## Scripts Explanation
2259

2360
This repository contains a few scripts used to hit the Enphase Enlighten API and collect data. The scripts inclide:
2461

2562
### run_inverter_daily_stats.py
2663

27-
Runs the Enlighten API route 'inverters_summary_by_envoy_or_site' to collect the lifetime energy produced by each inverter. If you call this route once a day before your solar is producing power (e.g.: 4am) you get the total lifetime power produced by each inverter, including the previous day. If you track this total lifetime energy value every day, you can then subtract the current day's total from the previous day lifetime total. That gives you the daily production value for that inverter. Note: if your Envoy is connected via low bandwidth Cellular, data only refreshes to Enlighten every 6 hours. So perform this route the next day in the early morning to ensure you get complete data.
64+
Runs the Enlighten API route 'inverters_summary_by_envoy_or_site' to collect the lifetime energy produced by each inverter. The Enphase API lacks the granulatiry of seeing per inveter daily states (documented here: https://developer.enphase.com/forum/topics/per-inverter-stats). So this script provides a means to do that. If you call this route once a day before your solar is producing power (e.g.: 4am) you get the total lifetime power produced by each inverter, including the previous day. If you track this total lifetime energy value every day, you can then subtract the current day's total from the previous day lifetime total. That gives you the daily production value for that inverter. Note: if your Envoy is connected via low bandwidth Cellular, data only refreshes to Enlighten every 6 hours. So perform this route the next day in the early morning to ensure you get complete data.
2865

2966
The resulting data is stashed in a .json file. The file organizes the data by microinverter (by ID), then by day. So you can easily parse this historical data for daily production values.
3067
For example:
@@ -61,14 +98,10 @@ For example:
6198
}
6299
}
63100

64-
65-
## Setting Automated Cron Jobs
66-
If you're using Linux, you can add these scripts to crontab jobs to run automatically at night by:
67-
68-
Run:
69-
70-
>crontab -e
71-
Add a crontab job:
72-
73-
# At 4am local time run the python script via shell script to ensure we're in the right directory
74-
0 4 * * * /home/<user>/python_enlighten_api/run_inverter_daily_stats.sh >> /home/<user>/python_enlighten_api/cron.log 2>&1
101+
### populat_google_sheet.py
102+
Is run after the Englighten API data has been captured to the specified sheet by:
103+
1. Grab all inverter serial numbers from the linked google sheet's Named Range 'InverterSerialNumbers'
104+
2. Load in captured enlighten historical data (from run_inverter_daily_stats.py)
105+
3. Look for the sheet titled 'Panel Data-<Current Year>' or duplicate it from 'Panel Data-Template' if it's not found
106+
4. Match up the Enlighten serial number list order vs the InverterSerialNumber range data and filter the data by the current day
107+
5. Insert the data into the 'Panel Data-<Current Year>' sheet

‎populate_google_sheet.py‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
import datetime
2+
import json
3+
import os
4+
from utils.googleSheetsAPI import googleSheetsAPI
5+
6+
INVERTER_STARTING_CELL = 'A3'
7+
8+
def run(config):
9+
SPREADSHEET_ID = config["spreadsheet_id"]
10+
print('Beginning push to Google Sheets...')
11+
api = googleSheetsAPI()
12+
13+
# Get our inverter serial numbers from our google sheet
14+
inverter_sns = []
15+
result_values = api.readRange(SPREADSHEET_ID, 'InverterSerialNumbers')
16+
for row in result_values:
17+
for cell in row:
18+
#print(cell)
19+
inverter_sns.append(cell)
20+
21+
# Grab the historical data from the data store
22+
inverter_historical_data = {}
23+
datafile = f'data/inverter_daily_data-{config["site_id"]}.json'
24+
if os.path.isfile(datafile) and os.access(datafile, os.R_OK):
25+
with open(datafile) as json_file:
26+
inverter_historical_data = json.load(json_file)
27+
else:
28+
print(f'Missing {datafile}')
29+
exit(1)
30+
31+
yesterday = datetime.datetime.now() + datetime.timedelta(days=-1)
32+
daily_data_to_populate = yesterday.strftime('%Y-%m-%d') #"2020-10-15"
33+
values = [ [yesterday.strftime("%m/%d/%Y")] ]
34+
35+
# Ensure we can find the sheet 'Panel Data-<YEAR>' to insert data into. If not, duplicate one
36+
# from the template sheet
37+
year = yesterday.strftime('%Y')
38+
# Check if the sheet for the current year exists. If not, make it
39+
sheets = api.getSheetList(SPREADSHEET_ID)
40+
target_sheet_found = False
41+
for sheet in sheets:
42+
if sheet['properties']['title'] == f'Panel Data-{year}':
43+
target_sheet_found = True
44+
break
45+
if target_sheet_found is False:
46+
template_sheet_id = api.getSheetId(SPREADSHEET_ID, "Panel Data-TEMPLATE")
47+
api.duplicateSheet(SPREADSHEET_ID, template_sheet_id, f'Panel Data-{year}', 6)
48+
49+
# Match up inverter serial numbers from the google sheet to
50+
for serial_num in inverter_sns:
51+
if serial_num in inverter_historical_data["micro_inverters"]:
52+
inverter_data = inverter_historical_data["micro_inverters"][serial_num]
53+
if daily_data_to_populate in inverter_data:
54+
values.append([inverter_data[daily_data_to_populate]["daily_energy"]])
55+
else:
56+
print(f'Date data: {daily_data_to_populate} missing from {datafile}')
57+
exit(1)
58+
else:
59+
print(f'Serial Number: {serial_num} missing from google sheet')
60+
exit(1)
61+
62+
body = {
63+
'majorDimension': 'COLUMNS',
64+
'values': values
65+
}
66+
# Add our daily data to our 'Panel Data' sheet
67+
api.appendDataToRange(SPREADSHEET_ID, f'Panel Data-{year}!{INVERTER_STARTING_CELL}', body)
68+
69+
if __name__ == '__main__':
70+
run(config)

‎run_inverter_daily_stats.py‎

Lines changed: 59 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -7,75 +7,75 @@
77
import requests
88
import os
99
from utils.enlightenAPI import enlightenAPI
10+
import populate_google_sheet
1011

1112
#-------------------------------------------
1213
# User Settings
1314
#-------------------------------------------
14-
user_id = '<user_id>'
15-
site_id = '<site_id>'
15+
configs = \
16+
[{
17+
"name": <some name>
18+
"user_id": <user_id>,
19+
"site_id": <site_id>,
20+
"api_url": <api_url>,
21+
"api_key": <api_key>
22+
"spreadsheet_id": <google sheet_id>
23+
}]
1624

17-
#-------------------------------------------
18-
# Enlighten Settings
19-
#-------------------------------------------
20-
api_url = 'https://api.enphaseenergy.com/api/v2/systems'
21-
api_key = '<api_key>'
22-
23-
config = \
24-
{
25-
"user_id": user_id,
26-
"site_id": site_id,
27-
"api_url": api_url,
28-
"api_key": api_key
29-
}
25+
for config in configs:
26+
print(f'Beginning EnlightenAPI pull for site {config["site_id"]}')
27+
api = enlightenAPI(config)
3028

31-
api = enlightenAPI(config)
29+
# Get the inverter data from the enlighten API
30+
inverter_summary = api.inverter_summary()[0]
3231

33-
# Get the inverter data from the enlighten API
34-
inverter_summary = api.inverter_summary()[0]
35-
36-
# Read in the existing data or create the folder/file if needed.
37-
# This should be formatted as:
38-
'''
39-
{
40-
"micro_inverters" : {
41-
"<ID>": {
42-
"<DATE>": {
43-
"daily_energy" <Wh>,
44-
"lifetime_energy": <Wh>
32+
# Read in the existing data or create the folder/file if needed.
33+
# This should be formatted as:
34+
'''
35+
{
36+
"micro_inverters" : {
37+
"<ID>": {
38+
"<DATE>": {
39+
"daily_energy" <Wh>,
40+
"lifetime_energy": <Wh>
41+
}
4542
}
4643
}
47-
}
48-
'''
49-
inverter_historical_data = {}
50-
if os.path.isfile("data/inverter_daily_data.json") and os.access("data/inverter_daily_data.json", os.R_OK):
51-
with open("data/inverter_daily_data.json") as json_file:
52-
inverter_historical_data = json.load(json_file)
53-
else:
54-
os.makedirs('data', exist_ok=True)
55-
inverter_historical_data["micro_inverters"] = {}
44+
'''
45+
inverter_historical_data = {}
46+
datafile = f'data/inverter_daily_data-{config["site_id"]}.json'
47+
if os.path.isfile(datafile) and os.access(datafile, os.R_OK):
48+
with open(datafile) as json_file:
49+
inverter_historical_data = json.load(json_file)
50+
else:
51+
os.makedirs('data', exist_ok=True)
52+
inverter_historical_data["micro_inverters"] = {}
53+
54+
# Load the inverter data to the dictionary.
55+
yesterday = (datetime.datetime.now() + datetime.timedelta(days=-1)).strftime('%Y-%m-%d')
56+
for inverter in inverter_summary["micro_inverters"]:
57+
inverter_sn = str(inverter["serial_number"])
58+
if inverter_sn not in inverter_historical_data["micro_inverters"]:
59+
inverter_historical_data["micro_inverters"][inverter_sn] = {}
60+
inverter_historical_data["micro_inverters"][inverter_sn][yesterday] = { "daily_energy": 0, "lifetime_energy": inverter["energy"]["value"]}
61+
62+
# Populate the daily_energy for each inverter for today's date based on the previous day's lifetime_energy and now's lifetime_energy
63+
two_days_ago = (datetime.datetime.now() + datetime.timedelta(days=-2)).strftime('%Y-%m-%d')
64+
total_daily_wh = 0
65+
for inverter_sn, inverter_data in inverter_historical_data["micro_inverters"].items():
66+
if two_days_ago in inverter_data:
67+
two_days_ago_lifetime_energy = inverter_data[two_days_ago]["lifetime_energy"]
68+
yesterday_lifetime_energy = inverter_data[yesterday]["lifetime_energy"]
69+
yesterday_energy = yesterday_lifetime_energy - two_days_ago_lifetime_energy
70+
inverter_historical_data["micro_inverters"][inverter_sn][yesterday]["daily_energy"] = yesterday_energy
71+
total_daily_wh = total_daily_wh + yesterday_energy
5672

57-
# Load the inverter data to the dictionary.
58-
yesterday = (datetime.datetime.now() + datetime.timedelta(days=-1)).strftime('%Y-%m-%d')
59-
for inverter in inverter_summary["micro_inverters"]:
60-
inverter_sn = str(inverter["serial_number"])
61-
if inverter_sn not in inverter_historical_data["micro_inverters"]:
62-
inverter_historical_data["micro_inverters"][inverter_sn] = {}
63-
inverter_historical_data["micro_inverters"][inverter_sn][yesterday] = { "daily_energy": 0, "lifetime_energy": inverter["energy"]["value"]}
73+
# Write new data to file
74+
with open(datafile, 'w') as outfile:
75+
json.dump(inverter_historical_data, outfile)
6476

65-
# Populate the daily_energy for each inverter for today's date based on the previous day's lifetime_energy and now's lifetime_energy
66-
two_days_ago = (datetime.datetime.now() + datetime.timedelta(days=-2)).strftime('%Y-%m-%d')
67-
total_daily_wh = 0
68-
for inverter_sn, inverter_data in inverter_historical_data["micro_inverters"].items():
69-
if two_days_ago in inverter_data:
70-
two_days_ago_lifetime_energy = inverter_data[two_days_ago]["lifetime_energy"]
71-
yesterday_lifetime_energy = inverter_data[yesterday]["lifetime_energy"]
72-
yesterday_energy = yesterday_lifetime_energy - two_days_ago_lifetime_energy
73-
inverter_historical_data["micro_inverters"][inverter_sn][yesterday]["daily_energy"] = yesterday_energy
74-
total_daily_wh = total_daily_wh + yesterday_energy
77+
print(f'Yesterdays\'s Total Energy: {total_daily_wh}Wh')
7578

76-
# Write new data to file
77-
with open('data/inverter_daily_data.json', 'w') as outfile:
78-
json.dump(inverter_historical_data, outfile)
79+
populate_google_sheet.run(config)
7980

80-
print(f'Yesterdays\'s Total Energy: {total_daily_wh}Wh')
81-
print('Complete...')
81+
print('Complete...')

‎solar_performance_example.png‎

99.2 KB
Loading

0 commit comments

Comments
 (0)