To send data from any device or any system to Edge Impulse you will need to deliver the data in the Data acquisition format. This is a small specification that describes the type of data, the sensor data itself, and information about the device that generated the data. You can optionally sign this data so you have cryptographic proof on the origin and integrity of the data. If you have your data already in CSV, JPG, PNG or WAV format, you can also just upload the data either using the uploader feature in the Studio, or using the CLI, see CLI Uploader for more information. For CSV files, either use CSV data acquisition specification or use the CSV Wizard feature for more flexibility. If you are using an Edge AI hardware with an Edge Impulse firmware it will already send data in this format, and if you want to collect data from other devices the data forwarder will automatically convert and sign your data.Documentation Index
Fetch the complete documentation index at: https://docs.nordic.edgeimpulse.com/llms.txt
Use this file to discover all available pages before exploring further.
Data acquisition format specification
The Edge Impulse Data Acquisition format is an optimized binary format for time-series data. It allows cryptographic signing of the data when sampling is complete to prove the authenticity of the data. Data is encoded using CBOR or JSON and signed according to the JSON Web Signature specification. Example in JSON:Fields
protected- information about the signature format.ver- alwaysv1(required).alg- the algorithm used to sign this file. EitherHS256(HMAC-SHA256) ornone(required).iat- date when the file was created in seconds since epoch. Only set this when the device creating the file has an accurate clock (optional).
signature- cryptographic signature for this file (required).payload- sensor data.device_name- unique identifier for this device. Only set this when the device has a globally unique identifier (e.g. MAC address). If this field is set the device shows up on the ‘Devices’ page in the studio (optional).device_type- device type, for example the exact model of the device. Should be the same for all similar devices (required).interval_ms- the frequency of the data in this file (in milliseconds). E.g. for 100Hz fill in10(new data every 10 ms.). You can use a float here if you need the precision (required). If you have sensors with different sampling frequencies, you should upscale to the highest frequency to keep the finest granularity. Example: sensor A is 100 Hz, sensor B is 5 Hz. Upscale sensor B to 100 Hz by duplicating each value 20 times (100/5). You could also smooth values over between samples.sensors- array with sensor axes.name- name of the axis.units- type of data on this axis. Needs to comply to SenML units (required).
values- array of sensor values. One array item per interval, and as many items in this array as there are sensor axes. If you have a single sensor, you are allowed to flatten this array to save space. An array of string values can be used to refer to binary attachments sent to the ingestion service using multipart requests.
Signing data
Files can be signed to establish a trust chain between device and Edge Impulse. Because theiat and device_name fields are included in the signed file you can validate authenticity of the data, which device created the data, and when the data was captured. Currently data can be signed with a symmetric HMAC key. You can create HMAC keys under ‘Dashboard’ in the studio.
HMAC SHA256
To sign data using HMAC SHA256:- Set the
signaturefield to0000000000000000000000000000000000000000000000000000000000000000(64 times ASCII0). - Create the full CBOR or JSON object.
- Use your language’s crypto module to sign the object with HMAC-SHA256. This should give you a 32 byte signature.
- Replace the empty signature with the HEX encoded signature (64 characters ASCII).
Binary notation (CBOR)
This is the same example as above, but in binary. Paste it into the CBOR playground to decode:Examples
Data can easily be generated by any language that has a CBOR or JSON library available.x-label header parameter when creating your request. See the header parameters available in the Ingestion API documentation.
Binary payloads (e.g. for audio)
For some payloads, like audio data, it’s not feasible to convert the raw sensor data into CBOR values while capturing data on the device. For these cases you can send a binary payload. The same header structure as above applies, but instead of writing the values in thevalues CBOR field you can append a binary array at the end of the file (in uint8, uint16, uint32, int8, int16, int32 or float32 format). To do so:
- Limit your data to a single sensor.
- Set the value of the
valuesfield to:[ "Ref-Binary-i16" ](so a string within an array, wherei16indicatesint16, alternatively you can useu16foruint16orf32forfloat32). - Write the CBOR header. Note that this needs to be a valid CBOR structure, and that it requires termination with
0xFF. The easiest to achieve this is by writing thevaluesfield as an indefinite length array (which needs to be terminated by0xFF. - Write your payload (little endian).
- If you need to align your writes the easiest is to pad the
Ref-Binary-i16string with spaces on the right.
- If you need to align your writes the easiest is to pad the
- Update the signature with the signature for the full file, including header and payload.
- Upload the data to the ingestion service with the
Content-Type: application/octet-streamheader.
ff (you can copy the beginning of the data up until the ff into cbor.me to validate) and the payload starts at f918fa.
Binary attachments (e.g. for images)
For non-time-series data - currently only implemented for images - you can send binary attachments to the ingestion service. Here the same header structure is used as above, but instead of writing the values in thevalues CBOR field, you reference the content type, the size, and the signed hash (with the same HMAC key) of your attachment. You then need to send both the header and the actual file to the Ingestion API via a multipart/form request (see ‘Multipart requests’ section).
Note: Including more than 1 binary attachment is currently not supported by the Edge Impulse Studio, such messages won’t be rejected by the ingestion service though, these attachments will merely be ‘invisible’ from the studio.