A test report that reads "delivered: 97%" is one of the most comforting and least informative numbers in a continuity program. It tells you the platform handed your message to a set of carriers and mail servers. It does not tell you that a person saw it, understood it, or did what it asked. Most organizations test the first thing and assume the second.
Mass notification gets used in the first five minutes of almost every incident. It is also the tool most likely to be configured once, handed to whoever sat nearest, and tested by sending "This is a test" to everyone twice a year. That proves the subscription is paid. Here is what is worth testing instead, roughly in the order things break.
1. The contact data, which is the test before the test
Nearly every notification failure traces back to data. The platform reaches the numbers it has: usually a nightly HR feed plus whatever someone uploaded by spreadsheet two years ago.
Start with a reconciliation, not a message. Pull the record count from the notification platform and from the HR system on the same day and compare them by department and location. The gaps are predictable:
- Contractors, temps and vendor staff who work on your sites but never appear in the HR system.
- Leavers who stay active because the feed adds people but nobody built the delete.
- Work phones only. A desk phone is of little use at 2 a.m., or when its building is the problem. You want a personal mobile, given with consent and a clear statement of how it will be used.
- Location fields that say "Remote" for half the workforce, so you cannot target the people who live in the storm's path.
- Recycled numbers that send a stranger your evacuation notice.
Decide who owns each field: HR owns identity and employment status, employees own their personal contact details through a self-service profile, the continuity team owns groups and roles. Then set a freshness rule (say, HR changes reach the platform within 24 hours) and check it monthly against a few recent hires and leavers. If the integration has failed silently, you want to find out on a quiet Tuesday.
2. Channel diversity, because each channel fails differently
Text, voice, app push, email and desktop pop-ups are not five copies of one thing. Each depends on different infrastructure, which is the point: they should not all fail together.
| Channel | Depends on | Typical way it fails |
|---|---|---|
| SMS | Mobile carriers, sender vetting | Carrier filtering, throttling, the recipient having replied STOP |
| Voice call | Carrier routing, caller ID reputation | Labeled as likely spam, sent to voicemail |
| App push | App installed, signed in, notifications allowed | A new phone or OS update quietly resets permissions |
| Your mail gateway, sender authentication | Quarantined as bulk or phishing, read hours later | |
| Desktop alert | A working, logged-in corporate endpoint | Useless when the endpoints are the outage |
That last row stopped being theoretical on 19 July 2024, when a faulty CrowdStrike Falcon update left Windows machines around the world stuck in boot failures. Organizations whose first channel was a desktop pop-up, or whose people read alerts on a company laptop, found that the channel and the incident were the same thing. Map channels against your top scenarios and look for a shared single point of failure: the corporate network, the identity provider, the building's power.
Keep at least one channel that does not depend on the platform at all. A recorded status line, a public status page or a manager call tree feels old-fashioned until the vendor has its own bad day.
3. Deliverability: the carriers have opinions about your message
Texts sent by software are treated differently from texts between people. US carriers expect application-to-person traffic from approved senders: short codes, verified toll-free numbers, or ten-digit numbers vetted for business messaging (the "10DLC" regime, in which each brand and use case is recorded with a central industry body). Unvetted traffic gets throttled or blocked, and enforcement has tightened lately. Your vendor handles most of it, but ask three questions in writing:
- Which sender type carries our emergency texts, and is it vetted under our own brand or shared with the vendor's other customers?
- What is the throughput? A route that sends a few hundred messages a minute will take a long time to reach 20,000 people.
- What happens when a carrier filters a message? Do we see it in the delivery report, or does it show as "sent"?
Content matters too. Carrier filters look for phishing patterns: public link shorteners, urgent requests to "verify" something, unfamiliar numbers. That creates an awkward conflict in your own building. Your security awareness training teaches people to ignore unexpected texts that contain links and demand fast action. Your emergency message is exactly that. Fix it from both ends: publish the platform's sending numbers and email domain on the intranet, ask employees to save the notification contact card, and keep links out of the first message where you can.
Two smaller traps. Calls from numbers with no reputation get labeled as probable spam, so keep the caller ID consistent and ask the vendor how its calls are authenticated. And an employee who replies STOP to an emergency text has, in most setups, opted out of all future texts from that sender until they reply START. Pull the opt-out list monthly and follow up person by person.
On email, make sure the vendor's sending domain is authenticated (SPF, DKIM and DMARC) and allowlisted by your own gateway. The large mailbox providers tightened bulk-sender rules in early 2024, and corporate gateways are often stricter still.
4. Templates and the 160-character problem
A standard SMS segment holds 160 characters, but only if every character comes from the basic GSM alphabet. One curly apostrophe pasted from a word processor, one long dash, one emoji, and the whole message switches to Unicode encoding, where a segment holds 70. A 150-character message becomes three segments, costs more, and on some routes arrives split or out of order. Check every template for encoding before it goes into the library.
Write templates for the first message, the update and the all-clear, for each common scenario: building evacuation, shelter in place, office closure, IT outage, severe weather. A first message needs five things: who it is from, what happened, where, what to do, and when the next update comes.
[Org] ALERT: Fire alarm at [building]. Evacuate now to [assembly point]. No elevators. Reply 1 if safe, 2 if you need help. Next update [time].
Listen to how text-to-speech reads your voice templates; acronyms and street abbreviations come out badly. And keep a pre-approved correction template. On 13 January 2018, a ballistic missile alert went out in error to phones across Hawaii, and the official correction took 38 minutes to reach the same phones, in part because no correction message had been prepared in advance. A false alarm without a fast retraction damages trust for years.
5. Who can send, and from where
Write down who may send, to which audiences, for what purposes, and who approves. Then test the people, not only the platform.
- Every authorized sender should send a real message to a test group each quarter, from a phone, away from the office. People forget passwords and menus between incidents.
- Check whether sender logins depend on your single sign-on. If the identity provider is down, which is plausible in a cyber incident, can anyone log in? Keep a few break-glass accounts with separately stored credentials, and test them.
- Require a second person for organization-wide sends, but give the on-site manager authority to send a building evacuation without waiting for anyone.
- Know how to send if the web console is unreachable: a mobile app, a phone-in option, or the vendor's support line with a verification process agreed in advance.
6. Two-way polling and accountability
OSHA's emergency action plan rule requires procedures to account for employees after an evacuation. A polling message is often the fastest way to do that across many sites, but only if the poll is built for the reply you need. Keep it to one question with numeric answers. "Reply 1 if safe, 2 if you need help, 3 if safe but unable to work" gives you safety and staffing in a single pass.
The real work is the non-responders. Decide in advance how long you wait, who chases them (usually their manager), and when silence becomes a welfare check. This is where duty of care obligations stop being an HR policy and become a list someone works through at night. If your crisis team runs from something like an emergency operations center, put the accountability count on the main status board next to the incident objectives.
Employees will also receive public alerts through Wireless Emergency Alerts and broadcast, and your message should not contradict them. Firms with a working relationship with their county emergency management office, the kind of public–private coordination that pays off in regional events, find it far easier to stay aligned on evacuation zones and shelter-in-place guidance.
The test plan
A starting point. Set the pass thresholds yourself and keep the results so you can see the trend.
| Test | What it proves | Frequency | Pass looks like |
|---|---|---|---|
| HR-to-platform reconciliation | Data is complete and current | Monthly | Counts match by site; leavers removed within your freshness rule |
| Template check | Messages fit one segment and read well aloud | Every template change | No Unicode characters; voice playback reviewed |
| Single-channel test, each channel | Each route works on its own | Quarterly | Receipt confirmed on sample handsets, not just in the report |
| Announced all-channel test with poll | End-to-end reach and response | Twice a year | Response rate at 15, 30 and 60 minutes meets your target |
| Unannounced after-hours poll | Reach when people are asleep or off-site | Annually | Non-responder escalation finished within the agreed time |
| Sender drill | Each sender can send from a phone, off-site | Quarterly | Every sender succeeds without help |
| SSO-down login | Break-glass access works | Twice a year | Login and send with the identity provider bypassed |
| Throughput test | Large groups finish in acceptable time | Annually and after vendor changes | Time to 90 percent delivered within target |
| Correction drill | A wrong message can be retracted fast | Annually | Correction delivered within minutes |
| Fallback channel check | Status line or page works without the platform | Quarterly | Updated and verified by someone outside the team |
Metrics worth reporting upward
Leadership does not need delivery percentages. Report the numbers that predict a bad night:
- Time to 90 percent confirmed, not delivered.
- Response rate at 15, 30 and 60 minutes on polling tests, trended over time.
- Unreachable people by department: no valid mobile, opted out, or bouncing.
- Data lag: how long, on average, an HR change takes to reach the platform.
- Sender readiness: authorized senders who completed a drill this quarter.
A falling response rate is often the first sign of a data problem, or of people quietly treating your tests as noise. One well-explained test with a visible result does more than four routine ones.
A first-quarter plan if you are starting over
- Weeks 1–2: Reconcile the HR feed against the platform. Fix contractors, leavers and location fields. Pull the opt-out list.
- Weeks 3–4: Get written vendor answers on sender vetting, throughput and filtered messages. Authenticate the email domain.
- Weeks 5–6: Rewrite the templates, check encoding, add correction and all-clear messages.
- Weeks 7–8: Rebuild the sender list. Run the first sender drill and the SSO-down login test.
- Weeks 9–10: Publish the sending numbers. Brief the security awareness team so the two programs stop contradicting each other.
- Weeks 11–13: Run an announced, all-channel test with a poll. Work the non-responder list to the end and record every reason someone did not get the message.
That last list is the real test result. Keep it, fix what is on it, and run the test again.
