How we score and where our data comes from
Every number we publish is either from a source we cite or computed by a rubric we publish. Here are the rubrics.
Principles#
- Reproducible over impressive. Every score has to be recomputable by a reader with the same public inputs. That rules out anything based on private testing.
- Inputs are facts, weights are opinions. We separate the two. You can accept our inputs and rebuild the ranking with your own weights, and the datasets are published so you can.
- Unknown is not zero. Where we cannot verify a criterion, it is marked unknown and excluded from the denominator rather than scored as a failure.
- Versioned. Datasets carry a version and a date. When a rubric changes, the change is noted here rather than applied silently.
The Local Control Index#
Question it answers: if your broadband goes down at 7pm, how much of your smart home still works?
Each platform is scored on eight criteria, worth 100 points in total. Scoring is per criterion: full points if the vendor documents the behavior, half points if it works but with a documented caveat, zero if it requires the cloud.
| Criterion | Points | What earns full points |
|---|---|---|
| Local device control from the app on the same LAN | 20 | App can turn devices on and off with no internet at all |
| Local automation execution | 20 | Schedules and triggers run on local hardware, not a server |
| Local voice control | 10 | Wake word, speech recognition and command execution all on device or on the LAN |
| Local state and sensor reporting | 10 | Sensor events reach the controller and update state without a round trip |
| Works after vendor shutdown | 15 | Documented offline or self-hosted mode that survives the company disappearing |
| Local API or integration surface | 10 | Documented LAN API, MQTT, or open integration that third parties can use |
| Local firmware and backup control | 10 | You can back up the configuration and update firmware without cloud dependency |
| No account required for basic operation | 5 | Setup and daily use possible without a vendor account |
The score is deliberately about capability, not intent: a platform that can run locally scores well even if most owners use it in cloud mode. Read the current results in the Local Control Index, and the underlying JSON is at [/data/local_control.json](/data/local_control.json).
The Matter feature gap tracker#
Question it answers: the specification says the feature exists, but can you use it?
For each Matter release we record: the release date, the device types and features it introduced, and then, for each major controller, whether that controller exposes the feature to users. Three states are used:
- Shipping: documented by the platform as available to end users.
- Partial: available with a documented limitation (a subset of attributes, one device class only, beta channel, or a specific hub model).
- Not yet: no vendor documentation showing user-facing support.
The gap for a feature is the number of months between the specification's publication and the date the first major controller shipped support. See the tracker.
The exit cost model#
Question it answers: if you left this platform tomorrow, what would you have to rebuy?
For a given inventory of devices, each device is placed in one of four classes:
- Portable: standards-based and multi-admin capable, moves to a new controller with no purchase (most Matter and Z-Wave devices).
- Portable with a bridge: works elsewhere only through a hub or bridge you already own or must buy once.
- Re-pairable but degraded: moves, but loses features (vendor-specific scenes, camera events, firmware updates).
- Stranded: tied to the platform or its cloud, no documented path out.
The exit cost calculator applies this to your device list and returns a hardware cost range plus the number of devices that would need re-pairing. Prices used are typical street prices in USD, updated with the dataset, and are indicative only.
Data collection and updates#
Inputs come from the sources ranked in our editorial standards: specifications first, certification databases second, vendor documentation third. Each dataset row carries the date it was last verified. Rows we could not verify in the current cycle keep their previous date, so you can see what is fresh.
Datasets are published as JSON at stable URLs and are licensed under Creative Commons Attribution 4.0, so you may reuse them with credit and a link.
Known limitations#
- Regional variation. Platform features and device availability differ by country. Our data reflects United States availability, frequencies and street prices unless a page says otherwise.
- Firmware drift. A platform can add or remove a capability in a point release. The index is a snapshot with a date on it.
- Selection. We track the platforms and device categories with meaningful installed bases. Niche or discontinued systems appear only where they illustrate a point.