Skip to content
This repository has been archived by the owner on Jun 7, 2022. It is now read-only.

Commit

Permalink
Merge pull request #55 from ReagentX/develop
Browse files Browse the repository at this point in the history 
Release/1.2
  • Loading branch information
@ReagentX
ReagentX committed Sep 19, 2020
2 parents d996281 + 410bc6b commit d4172ff
Show file tree
Hide file tree
Showing 15 changed files with 690 additions and 260 deletions.
22 changes: 19 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Purple Air API
# PurpleAir API

A Python 3.x module to turn data from the PurpleAir/ThingSpeak API into a Pandas DataFrame safely, with many utility methods.
A Python 3.x module to turn data from the PurpleAir/ThingSpeak API into a Pandas DataFrame safely, with many utility methods and clear errors.

![Global Sensor Map with Celsius Scale](maps/sensor_map.png)

Expand Down Expand Up @@ -48,7 +48,9 @@ print(s) # Sensor 2891 at 10834, Canyon Road, Omaha, Douglas County, Nebraska,
```python
from purpleair.network import SensorList
p = SensorList() # Initialized 10,812 sensors!
df = p.to_dataframe(sensor_filter='all' channel='a') # Other options include 'outside' and 'useful'
# Other sensor filters include 'outside', 'useful', 'family', and 'no_child'
df = p.to_dataframe(sensor_filter='all',
channel='parent')
```

Result:
Expand All @@ -63,6 +65,20 @@ id
53069 47.190197 -122.177992 #2 outside 109.82 79.0 26.111111 ... False 0 109.52 108.72 109.33 116.64 74.52
```

### Make a DataFrame from all current sensors that have a 10 minute average pm2.5 value

```python
from purpleair.network import SensorList
p = SensorList() # Initialized 10,812 sensors!
# If `sensor_filter` is set to 'column' then we must also provide a value for `column`
df_1 = p.to_dataframe(sensor_filter='all',
channel='parent')
df_2 = p.to_dataframe(sensor_filter='column',
channel='parent',
column='m10avg') # See Channel docs for all column options
print(len(df_1), len(df_2)) # 11,071 10,723
```

### Get historical data for parent sensor secondary channel

```python
Expand Down
115 changes: 114 additions & 1 deletion docs/api/channel_methods.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,120 @@

## `setup()`

This converts the JSON metadata to Python class members, exposing data in a Pythonic way.
This converts the JSON metadata to Python class properties, exposing data in a Pythonic way.

## `as_dict() -> dict`

Return a dictionary representation of a Channel. The data is shaped like this:

```python
{
'meta': {
'id': a.identifier,
'parent': None,
'lat': a.lat,
'lon': a.lon,
'name': a.name,
'location_type': a.location_type
},
'data': {
'pm_2.5': a.current_pm2_5,
'temp_f': a.current_temp_f,
'temp_c': a.current_temp_c,
'humidity': a.current_humidity,
'pressure': a.current_pressure,
'p_0_3_um': a.current_p_0_3_um,
'p_0_5_um': a.current_p_0_5_um,
'p_1_0_um': a.current_p_1_0_um,
'p_2_5_um': a.current_p_2_5_um,
'p_5_0_um': a.current_p_5_0_um,
'p_10_0_um': a.current_p_10_0_um,
'pm1_0_cf_1': a.current_pm1_0_cf_1,
'pm2_5_cf_1': a.current_pm2_5_cf_1,
'pm10_0_cf_1': a.current_pm10_0_cf_1,
'pm1_0_atm': a.current_pm1_0_atm,
'pm2_5_atm': a.current_pm2_5_atm,
'pm10_0_atm': a.current_pm10_0_atm
},
'diagnostic': {
'last_seen': a.last_seen,
'model': a.model,
'hidden': a.hidden,
'flagged': a.flagged,
'downgraded': a.downgraded,
'age': a.age,
'brightness': a.brightness,
'hardware': a.hardware,
'version': a.version,
'last_update_check': a.last_update_check,
'created': a.created,
'uptime': a.uptime,
'is_owner': a.is_owner
},
'statistics': {
'10min_avg': a.m10avg,
'30min_avg': a.m30avg,
'1hour_avg': a.h1ravg,
'6hour_avg': a.h6ravg,
'1day_avg': a.d1avg,
'1week_avg': a.w1avg
}
}
```

## `as_flat_dict() -> dict`

Returns a flat dictionary representation of the Channel data.

The data is shaped like this:

```python
{
'parent': 0,
'lat': 0,
'lon': 0,
'name': 0,
'location_type': 0,
'pm_2.5': 0,
'temp_f': 0,
'temp_c': 0,
'humidity': 0,
'pressure': 0,
'p_0_3_um': 0,
'p_0_5_um': 0,
'p_1_0_um': 0,
'p_2_5_um': 0,
'p_5_0_um': 0,
'p_10_0_um': 0,
'pm1_0_cf_1': 0,
'pm2_5_cf_1': 0,
'pm10_0_cf_1': 0,
'pm1_0_atm': 0,
'pm2_5_atm': 0,
'pm10_0_atm': 0,
'last_seen': 0,
'model': 0,
'adc': 0,
'rssi': 0,
'hidden': 0,
'flagged': 0,
'downgraded': 0,
'age': 0,
'brightness': 0,
'hardware': 0,
'version': 0,
'last_update_check': 0,
'created': 0,
'uptime': 0,
'is_owner': 0,
'10min_avg': 0,
'30min_avg': 0,
'1hour_avg': 0,
'6hour_avg': 0,
'1day_avg': 0,
'1week_avg': 0
}
```

## `get_historical(weeks_to_get: int, thingspeak_field: str) -> pd.DataFrame`

Expand Down
103 changes: 61 additions & 42 deletions docs/api/sensor_methods.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -164,68 +164,84 @@ Return a dictionary representation of a sensor. The data is shaped like this:
'30min_avg': a.m30avg,
'1hour_avg': a.h1ravg,
'6hour_avg': a.h6ravg,
'1day_avg': a.d1avg,
'1week_avg': a.w1avg
}
},
'child':{
'meta': {
'id': a.identifier,
'parent': None,
'lat': a.lat,
'lon': a.lon,
'name': a.name,
'location_type': a.location_type
'id': b.identifier,
'parent': a.identifier,
'lat': b.lat,
'lon': b.lon,
'name': b.name,
'location_type': b.location_type
},
'data': {
'pm_2.5': a.current_pm2_5,
'temp_f': a.current_temp_f,
'temp_c': a.current_temp_c,
'humidity': a.current_humidity,
'pressure': a.current_pressure,
'p_0_3_um': a.current_p_0_3_um,
'p_0_5_um': a.current_p_0_5_um,
'p_1_0_um': a.current_p_1_0_um,
'p_2_5_um': a.current_p_2_5_um,
'p_5_0_um': a.current_p_5_0_um,
'p_10_0_um': a.current_p_10_0_um,
'pm1_0_cf_1': a.current_pm1_0_cf_1,
'pm2_5_cf_1': a.current_pm2_5_cf_1,
'pm10_0_cf_1': a.current_pm10_0_cf_1,
'pm1_0_atm': a.current_pm1_0_atm,
'pm2_5_atm': a.current_pm2_5_atm,
'pm10_0_atm': a.current_pm10_0_atm
'pm_2.5': b.current_pm2_5,
'temp_f': b.current_temp_f,
'temp_c': b.current_temp_c,
'humidity': b.current_humidity,
'pressure': b.current_pressure,
'p_0_3_um': b.current_p_0_3_um,
'p_0_5_um': b.current_p_0_5_um,
'p_1_0_um': b.current_p_1_0_um,
'p_2_5_um': b.current_p_2_5_um,
'p_5_0_um': b.current_p_5_0_um,
'p_10_0_um': b.current_p_10_0_um,
'pm1_0_cf_1': b.current_pm1_0_cf_1,
'pm2_5_cf_1': b.current_pm2_5_cf_1,
'pm10_0_cf_1': b.current_pm10_0_cf_1,
'pm1_0_atm': b.current_pm1_0_atm,
'pm2_5_atm': b.current_pm2_5_atm,
'pm10_0_atm': b.current_pm10_0_atm
},
'diagnostic': {
'last_seen': a.last_seen,
'model': a.model,
'hidden': a.hidden,
'flagged': a.flagged,
'downgraded': a.downgraded,
'age': a.age,
'brightness': a.brightness,
'hardware': a.hardware,
'version': a.version,
'last_update_check': a.last_update_check,
'created': a.created,
'uptime': a.uptime,
'is_owner': a.is_owner
'last_seen': b.last_seen,
'model': b.model,
'adc': b.adc,
'rssi': b.rssi,
'hidden': b.hidden,
'flagged': b.flagged,
'downgraded': b.downgraded,
'age': b.age,
'brightness': b.brightness,
'hardware': b.hardware,
'version': b.version,
'last_update_check': b.last_update_check,
'created': b.created,
'uptime': b.uptime,
'is_owner': b.is_owner
},
'statistics': {
'10min_avg': a.m10avg,
'30min_avg': a.m30avg,
'1hour_avg': a.h1ravg,
'6hour_avg': a.h6ravg,
'1week_avg': a.w1avg
'10min_avg': b.m10avg,
'30min_avg': b.m30avg,
'1hour_avg': b.h1ravg,
'6hour_avg': b.h6ravg,
'1day_avg': b.d1avg,
'1week_avg': b.w1avg
}
}
}
```

## `as_list() -> dict`

Return a list representation of a sensor. The data is shaped the same as `as_dict` except instead of `parent` and `child` keys, the `parent` is the 0th index and the child is the 1st index:

```python
[
{...parent_sensor_data...},
{...child_sensor_data...},
]
```


## `as_flat_dict(channel: str) -> dict`

Returns a flat dictionary representation of the Sensor data.

`channel` is one of `{'a', 'b'}`.
`channel` is one of `{'parent', 'child'}`.

The data is shaped like this:

Expand Down Expand Up @@ -255,6 +271,8 @@ The data is shaped like this:
'pm10_0_atm': 0,
'last_seen': 0,
'model': 0,
'adc': 0,
'rssi': 0,
'hidden': 0,
'flagged': 0,
'downgraded': 0,
Expand All @@ -270,6 +288,7 @@ The data is shaped like this:
'30min_avg': 0,
'1hour_avg': 0,
'6hour_avg': 0,
'1day_avg': 0,
'1week_avg': 0
}
```
14 changes: 12 additions & 2 deletions docs/api/sensorlist_methods.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,27 @@

Automatically run on instantiation. Retrieves the current network data from the PurpleAir API.

## `to_dataframe(sensor_group: str, channel: str) -> pd.DataFrame`
## `to_dataframe(sensor_group: str, channel: str, column: Optional[str] = None, value_filter: Union[str, int, float, None] = None) -> pd.DataFrame`

Converts dictionary representation of a list of sensors to a Pandas DataFrame where `sensor_group` determines which group of sensors are used.

`channel` is one of `{'a', 'b'}`.
`channel` is one of `{'parent', 'child'}`.

* `'useful'`
* Sensors with no faults, as determined by [`is_useful()`](/docs/api/sensor_methods.md#is_useful---bool)
* `'outside'`
* Outdoor sensors only
* `'all'`
* Do not filter sensors
* `family`
* Sensor has both parent and child
* `no_child`
* Sensor is parent-only
* `column`
* Must be a value that exists on a [Channel](/docs/documentation.md#channel)
* If `value_filter` is not provided:
* Sensor has data in `column`, i.e. no `None` values
* If `value_filter` is provided:
* Sensor has data in `column` that is the same as `value_filter`

If `sensor_group` is not in the above set, `to_dataframe()` will raise a `ValueError`.
12 changes: 5 additions & 7 deletions docs/documentation.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

There are two main components of this program: `SensorList` and `Sensor`. A `SensorList` instance represents the network or a subset of the network of PurpleAir sensors, while a `Sensor` represents a single sensor.

`Sensor`s have up to two channels, a `parent` and an optional `child`, that hold data the sensor generates.

## SensorList

PurpleAir sensor network representation
Expand All @@ -12,13 +14,9 @@ PurpleAir sensor network representation

To parse location of all sensors from coordinates to addresses, pass `SensorList(parse_location=True)`.

* Members
* Properties
* `all_sensors`
* All sensors in the PurpleAir network
* `outside_sensors`
* Outdoor sensors in the PurpleAir network
* `useful_sensors`
* Sensors without faults in the PurpleAir network

See [api/sensorlist_methods.md](api/sensorlist_methods.md) for method documentation.

Expand All @@ -36,7 +34,7 @@ Initialize a new sensor.

`parse_location` is an optional boolean parameter to use `geopy` to parse the rough address of the location of the sensor based on the latitude and longitude from the sensor's metadata.

* Members
* Properties
* `identifier`
* Sensor ID Number
* `data`
Expand Down Expand Up @@ -67,7 +65,7 @@ Representation of a sensor channel, either `a` or `b`. For channel `b` (child) s

### `class Channel(channel_data: dict)`

* Members
* Properties
* `channel_data`
* metadata in Python dictionary format about the channel
* `lat`
Expand Down
Loading

0 comments on commit d4172ff

Please sign in to comment.