Every IoT architecture answers one question many times over: for this piece of work, where does the code run? The sensor, the gateway in the cabinet, a server in the building, a node at the telecom edge, or a cloud region on another continent are all candidates, and each is the right answer for some workloads and the wrong one for others.
The vocabulary around this is messier than it needs to be. Cloud computing is clear enough: centralized, elastic, remote. Edge computing means processing near the data source, but “near” stretches from a microcontroller to a metro data center. Fog computing, a term promoted by Cisco and the OpenFog Consortium in the mid-2010s, describes the intermediate layers between devices and cloud, with compute, storage and networking distributed along the path. In practice most people now fold fog into “edge” and talk about tiers.
This guide lays out the tiers, the factors that decide placement, and a repeatable way to assign workloads.
The tiers, from thinnest to thickest
Device tier. The sensor or actuator itself, usually a microcontroller with kilobytes to a few megabytes of memory. It can filter, threshold, compress and run small models, and it is the only tier that can act with microsecond timing. It cannot store history or coordinate with its neighbors beyond a radio hop.
Gateway tier. An edge gateway with a full operating system, aggregating a site’s devices. It can translate protocols, buffer during outages, run rules and medium-sized models, and host a local user interface. It knows about the devices in its site and nothing beyond.
Site or fog tier. On-premises servers or a small cluster, or operator-hosted compute at the network edge. It can run a local historian, heavier inference, a site-level digital twin and coordination across many gateways. It has real storage and real compute, but it is still a fixed, finite resource that someone maintains on site.
Cloud tier. Elastic compute and storage with global reach. It is the only tier that can see the whole fleet, hold years of history, train models on cross-site data, and serve people who are not on site. It is also the farthest away and the one that disappears when the link does.
The factors that decide placement
Response time
Measure the deadline, not the preference. A control loop that must close in under ten milliseconds lives on the device or on a controller wired to it. A decision needed within a second can live on a gateway. Anything that tolerates seconds or minutes can live anywhere, including the cloud. Be honest about which category a workload falls into; many “real-time” dashboards are fine with a two-second delay.
Data volume versus link capacity
Compare what the workload produces with what the uplink can carry at an acceptable cost. Video, high-rate vibration and audio almost always need reduction before leaving the site. Low-rate telemetry over Ethernet or Wi-Fi can go straight up. Over LPWAN or metered cellular, even small streams add up, and downsampling at the edge pays for itself.
Behavior when disconnected
Decide what must keep working during an outage. Local safety, local control, local alarms and local buffering belong below the link. Fleet reporting and historical analysis can wait. The data pipeline should be designed so that an outage produces a backlog, not data loss.
Privacy, regulation and sovereignty
Identify data that cannot leave the site, the country or the customer’s control. Process it below that boundary and transmit only what is allowed. Data sovereignty obligations often decide the fog tier’s existence on their own.
Scope of knowledge
Some computations need information that only exists in one tier. Comparing this machine with the same model in forty other plants needs the cloud. Detecting that this machine’s bearing is wearing needs only its own history, which a gateway can hold. Match the workload to the narrowest tier that holds the data it needs.
Cost and operations
Cloud compute is billed per use and maintained by the provider; edge hardware is bought up front and maintained by you or your customer. A thousand gateways running a model is a thousand units of hardware, power and patching. Sometimes the cloud is cheaper because it exists already; sometimes the edge is cheaper because data transfer dominates the bill. Run the numbers for the total cost of ownership rather than assuming.
A worked example: a manufacturing line
Take a packaging line with vision inspection, vibration sensors on motors, a PLC per station, and a plant manager who wants an OEE dashboard.
- Emergency stops and interlocks: device tier, on the PLC. Deterministic, no network dependency.
- Defect detection from cameras: gateway or site tier. Frames are too heavy to upload, the decision is needed within the cycle time, and the images may contain proprietary product details. Only defect images and counts go upstream.
- Bearing wear detection from vibration: device tier for feature extraction (the sensor computes spectral bands), gateway tier for the anomaly detection model, cloud tier for training the model on data from all plants and pushing updated versions back.
- OEE dashboard: cloud tier, because the plant manager, the regional director and the vendor all need to see it, and a few seconds of delay is fine. A site-tier copy keeps the local screens alive when the link drops.
- Model training and fleet comparison: cloud tier, where history and compute are.
The same workload often spans tiers, with feature extraction low and learning high. That is normal and is how the AIoT pattern usually settles.
Common mistakes
Choosing a tier for the whole system instead of per workload is the first. “We are an edge company” or “we are cloud-first” both produce poor placements. The second is designing the happy path only, so the first outage reveals that the local gateway cannot act without a cloud token. The third is underestimating edge operations: the prototype runs on one box in the lab, and nobody has planned how to update five hundred of them in cabinets across a region. The fourth is treating fog as a product to buy rather than a placement decision to make; the middle tier is valuable exactly when a specific workload needs it, and dead weight otherwise.
A short decision procedure
For each workload, write down its deadline, its data volume, its disconnected behavior, its data constraints, the scope of data it needs, and the people who consume its output. Place it in the narrowest tier that satisfies the deadline and the data constraints and that holds the data it needs. Then check the cost and operations consequences, and move it up a tier if the lower one is not worth maintaining. Revisit when the link, the volumes or the regulations change.
Where to go next
- Edge computing explained covers the forces behind edge placement in more depth.
- IoT architecture: the layers gives the layered model these tiers map onto.
- The IoT data pipeline follows data through ingestion, storage, processing and action across the tiers.
- IoT platforms explained looks at the cloud and edge runtimes that implement this placement.
Topics: edge computing, cloud computing, fog computing, architecture, latency