Data ingestion
The stage of a pipeline that receives data from devices, validates and normalizes it, and hands it to storage or processing reliably and at scale.
Ingestion is where device traffic meets the backend. A broker or HTTP endpoint accepts the payload, the ingestion service checks it is well-formed, decodes it from the device’s compact format into something applications can use, attaches identity and timestamp metadata, and writes it to a message queue or directly to a database.
Two things make this harder than it sounds. Volume is bursty: a reconnecting fleet or a daily reporting window can multiply traffic in seconds, so ingestion needs buffering and back-pressure. And data is imperfect: duplicates, out-of-order messages and malformed readings from a firmware bug must be handled without dropping the good data. The IoT data pipeline walks through the stages one by one.
Part of the IoT Glossary. Spotted an error or a missing term? Tell us.