Qisense AI Technology CenterQISENSE / PHYSICAL INTELLIGENCECollaborate
Back to applications
SCENARIO DESIGN / 02

Multimodal fusion: giving events context

The on-site question

An isolated signal rarely determines the right response. An arrival, a dim room and an explicit user choice are different kinds of input. The system must understand how they combine in time and priority.

Interaction & system design

Organize inputs into event windows while retaining their sources and expiry times. Explicit actions take priority over passive inference. Conflicting signals lead to a low-interference state or a request for confirmation. Site operators should be able to explain and adjust these rules.

Validation approach

  1. List each input's validity window and expiry conditions.
  2. Test rule priorities when multiple events occur at once.
  3. Disconnect one input and check whether others are misinterpreted.
Discuss your space
DEPLOYMENT PLAYBOOK

Multimodal integration & event bus

Normalize device outputs into events before deciding whether to trigger a workflow. Each event carries a source, zone, time, type, validity period and quality flag. Treat position changes separately from environmental changes so duplicate reports are not mistaken for repeated visits.

Delivery stages & outputs

StageWhat to recordWhat to check
Device integrationTime synchronization, event format, heartbeat statusDevice outages are clearly visible
Event fusionTime windows, zone association, deduplicationDuplicate reports do not repeat actions
Device coordinationLighting, screens, audio or service interfacesOutputs can be canceled and acknowledge outcomes

Exception handling & manual override

Stop automatic rules that depend on an offline device while retaining baseline lighting and manual controls. Give manual overrides explicit priority and recheck device connectivity before restoring automation. Log false, repeated and missed triggers separately rather than masking zone configuration issues with increased sensitivity.

Delivery checklist

  • Zone and device inventory: location, purpose, connection method and maintenance owner.
  • Coordination rules: triggers, cooldowns, priorities and cancellation conditions.
  • Acceptance records: normal flows, exceptions and human takeover replays.
  • Operations manual: inspections, configuration backups, content updates and fault rollback.