How we test
How we test what we say
Every number on this site comes from a test we ran. Here is each test, how it works, what it found, what it got wrong and what it cannot tell you. These are our own tests, not customer results.
We planted 33 made-up private details and checked every one
This is a controlled test on a practice app built for it. Every detail is made up and every screen of the practice app says so. Here is what was planted, what each pass caught, what it covered by mistake and what the test cannot tell you.
Method
What we did
What was planted
33 made-up private details: 15 inside form fields and 18 as page text. Kinds: phone numbers, email addresses, passwords and PINs, BVN, card security codes, NIN, one-time codes, bank account numbers, card numbers and expiry dates, names, balances and amounts and home addresses.
Where
Practice Bank, a made-up bank app built for this test, with 8 screens. Every screen says “Practice app, made-up data”.
How it was recorded
The real StepLoom Recorder recorded 8 clicks in 22 s and saved 9 step pictures. StepLoom AI then read the same pictures in 26.6 s.
How it was scored
Hidden by the Recorder: at least 80 percent of the item's pixels are black in the saved picture. Found by StepLoom AI: one of its boxes covers at least 60 percent of the item and is no larger than 12 times its area.
Results
What each pass caught
Each planted detail was checked in the saved pictures. The Recorder covers before a picture is saved. StepLoom AI reads the saved pictures for what is still visible.
33 of 33
hidden by the Recorder or found by StepLoom AI
19 of 33
hidden by the Recorder before the picture was saved
14 of 14
found by StepLoom AI, of those the Recorder left visible
0 of 33
left readable after both passes
| Kind | Planted | Recorder hidHidden by the Recorder | StepLoom AI foundFound by StepLoom AI | Missed |
|---|---|---|---|---|
| Phone numbers | 4 | 4 | 0 | 0 |
| Email addresses | 3 | 3 | 0 | 0 |
| Passwords and PINs | 2 | 2 | 0 | 0 |
| BVN | 1 | 1 | 0 | 0 |
| Card security codes | 1 | 1 | 0 | 0 |
| NIN | 1 | 1 | 0 | 0 |
| One-time codes | 1 | 1 | 0 | 0 |
| Bank account numbers | 5 | 4 | 1 | 0 |
| Card numbers and expiry | 3 | 2 | 1 | 0 |
| Names | 5 | 0 | 5 | 0 |
| Balances and amounts | 4 | 0 | 4 | 0 |
| Home addresses | 3 | 0 | 3 | 0 |
| All kinds | 33 | 19 | 14 | 0 |
- By where it sat. The Recorder hid 11 of 15 details inside form fields and 8 of 18 written as page text.
- What it left. It hid none of the names, balances and amounts and home addresses. StepLoom AI found all 14 it left.
- Partly hidden. The Recorder hid 4 of 5 bank account numbers and 2 of 3 card numbers or expiry dates. The account number written with spaces and the card expiry stayed visible until StepLoom AI boxed them.
False alarms
What was covered that was not private
A good check hides what is private and leaves the rest. We planted harmless look-alikes and counted how many each pass covered.
Look-alikes planted
20 harmless things that look private: order numbers, dates, reference codes, a branch code, a version, a fee and a limit. 2 of them have the same digit shape as a real ID or account number (an 11 digit request id and a 10 digit tracking number).
Covered by the StepLoom Recorder
2 of 20. Both were the two with the shape of an ID or account number.
Boxed by StepLoom AI
3 of 20: the 11 digit request id, a transaction reference code and a fee. It drew 21 boxes in all; 0 were on nothing, and 2 were on details the Recorder had already hidden.
What a false alarm costs
A cover on something that is not private hides a little too much. Every cover is a box you can turn off in the editor before you publish.
| Harmless, but covered | By the Recorder | By StepLoom AI |
|---|---|---|
| request id, 11 digits (same shape as an ID number) | Yes | Yes |
| transaction reference code | No | Yes |
| a fee | No | Yes |
| tracking number, 10 digits (same shape as an account number) | Yes | No |
Limitations
- One practice app, 8 screens, one run and one pass of StepLoom AI. It was not repeated, so there is no spread, and 33 items is a small count.
- The data is made up and the layout is clean: Latin script, English only, a plain page. Real apps are messier.
- Text drawn inside an image or a canvas is not page text. This test had none, no pictures of cards or IDs, no QR codes and no scrolled pages, so it gives no result for them.
- The Recorder's rules go by shape and field name, so they hide what looks like a phone number, an email or an ID and miss what does not, such as names, balances and addresses. StepLoom AI covers that gap in this run, and it can miss things too.
- The scores depend on the rules above (80 percent, 60 percent and 12 times). Looser or stricter rules change the StepLoom AI numbers. We did not try others.
- These numbers describe this build on this app. They are not a rate for every app. Look at the covers on your own demo before you share it.
Every test behind a number on this site, with its method, is gathered on How we test.
Checked by a machine. Not yet read by a native speaker.
Every translation below went through an automatic check that looks for lost meaning. It is useful, and it is not a review by a person. This is what it does, what it found and what it cannot tell you.
No native speaker has reviewed these yet.
The numbers on this page come from an automatic check, and they are labelled that way. They do not tell you how a voice sounds, whether a sentence feels natural or whether an instruction is easy to follow. Only a native speaker can say that.
Automatic check
How the check works
Automatic back-translation check. This is a machine check, not a native review.
What was checked
76 captions: 19 in each of 4 languages (Nigerian Pidgin, Yoruba, Igbo, Hausa). They are the steps, cover and last screen of the two public demos.
1. Translate back
StepLoom AI translated each caption back into English. It saw only the translated text, not the original.
2. Score the meaning
A second StepLoom AI call compared that English with the original caption and scored the meaning from 1 (different meaning) to 5 (same meaning).
3. Screen names
Separately, the name of the button or link each step tells the viewer to click (such as Units or Weekly report) was checked for appearing unchanged, because the product keeps screen names as they are.
Results
What the check found
8 StepLoom AI calls in all. Each language has 19 captions, so the mean is the average of 19 whole-number scores.
| Language | Captions | Mean score of 5 | Below 4 | Screen names kept |
|---|---|---|---|---|
| Nigerian Pidgin | 19 | 4.79 | 0 | 12 of 12 |
| Yoruba | 19 | 4.79 | 0 | 12 of 12 |
| Igbo | 19 | 4.84 | 0 | 12 of 12 |
| Hausa | 19 | 4.74 | 1 | 12 of 12 |
| All | 76 | 1 | 48 of 48 |
Lowest score
The weakest caption scored 3 of 5
Hausa, last screen of the ClientClue demo. This is the caption the check liked least, shown as it was found.
Original English
Now try it on your own screens. This demo was recorded on ClientClue's public demo pages.
Translated back to English
Now try on your phone. This test was taken on ClientClue demo pages.
What went wrong
"on your screens" narrowed to "on your phone"; "demo" changed to "test".
Limitations
- It does not judge how a word is pronounced, whether a sentence sounds natural or whether an instruction is easy to follow.
- It does not hear the voices. In the two public demos only English, Pidgin and Yoruba have voices, and Igbo and Hausa are text only.
- The translations and the check both come from StepLoom AI, so the check can miss a mistake that the translation also made.
- 76 captions from two demos is a small sample. Every score is a whole number from 1 to 5, so a mean score is not a percent of sentences that are right.
- The next check is a native speaker reading each caption and hearing each voice. No review has been finished, so none is shown here.
Every test behind a number on this site, with its method, is gathered on How we test.
How fast and how small a demo is
We open each public demo in a real browser with the network slowed down, then count the time and the bytes. These are the setups, what was counted and every result.
Method
What we did
Two connections
Regular 3G is 750 kbps with 300 ms delay. Slow 3G is 400 kbps with 400 ms delay. The network was slowed with the browser's own tools, with the cache off.
Two screens
Phone: 390 pixels wide at 2x. Computer: 1280 pixels wide at 1x. The phone is a phone-sized window, not a real phone.
What is timed
First screen: from opening the link until the first step's picture is on screen. Every step: opening the link, pressing Start and clicking through every step until the last screen shows, with no pause added and sound off.
What is counted
Bytes that crossed the network: the page, the player's scripts and styles, and the pictures. Voice audio is counted on its own, in a second pass with one voice language on.
Results
What we measured
Across 4 demos, one run each. The same demo opened on each connection and each screen, so the bytes are the same on both connections and only the time changes.
1.0 to 1.5 s
first screen on a phone, regular 3G (750 kbps)
1.7 to 2.5 s
first screen on a phone, slow 3G (400 kbps)
1.4 to 1.5 s
first screen on a computer, regular 3G (750 kbps)
230 to 255 KB
the whole demo on a phone: page, player, styles and pictures
Filter MTN Nigeria's reviews by language and feeling in ClientClue
Made with the StepLoom Recorder. Whole demo on a phone: 238 KB with the page and player; pictures 184 KB; the Pidgin voice adds 270 KB.
| Connection and screen | First screen | Every step | Whole demo |
|---|---|---|---|
| Regular 3G (750 kbps)Phone, 390 px wide | 1.0 s | 5.3 s | 238 KB |
| Regular 3G (750 kbps)Computer, 1280 px wide | 1.4 s | 6.7 s | 344 KB |
| Slow 3G (400 kbps)Phone, 390 px wide | 1.7 s | 8.7 s | 238 KB |
| Slow 3G (400 kbps)Computer, 1280 px wide | 2.3 s | 11 s | 344 KB |
Read MTN Nigeria's weekly report in ClientClue on an iPhone
Made from a phone recording. Whole demo on a phone: 245 KB with the page and player; pictures 196 KB; the Pidgin voice adds 143 KB.
| Connection and screen | First screen | Every step | Whole demo |
|---|---|---|---|
| Regular 3G (750 kbps)Phone, 390 px wide | 1.5 s | 4.2 s | 245 KB |
| Regular 3G (750 kbps)Computer, 1280 px wide | 1.5 s | 4.1 s | 245 KB |
| Slow 3G (400 kbps)Phone, 390 px wide | 2.5 s | 6.7 s | 245 KB |
| Slow 3G (400 kbps)Computer, 1280 px wide | 2.5 s | 6.8 s | 245 KB |
Explore AbujaMart's real orders in SheetScope
Made from uploaded screenshots. Whole demo on a phone: 255 KB with the page and player; pictures 202 KB; the Pidgin voice adds 256 KB.
| Connection and screen | First screen | Every step | Whole demo |
|---|---|---|---|
| Regular 3G (750 kbps)Phone, 390 px wide | 1.2 s | 5.5 s | 255 KB |
| Regular 3G (750 kbps)Computer, 1280 px wide | 1.5 s | 8.1 s | 403 KB |
| Slow 3G (400 kbps)Phone, 390 px wide | 1.9 s | 9.1 s | 255 KB |
| Slow 3G (400 kbps)Computer, 1280 px wide | 2.6 s | 13 s | 403 KB |
Read MTN Nigeria's reviews in ClientClue's workspace tour
Made from uploaded screenshots. Whole demo on a phone: 230 KB with the page and player; pictures 179 KB; the Pidgin voice adds 246 KB.
| Connection and screen | First screen | Every step | Whole demo |
|---|---|---|---|
| Regular 3G (750 kbps)Phone, 390 px wide | 1.1 s | 4.9 s | 230 KB |
| Regular 3G (750 kbps)Computer, 1280 px wide | 1.4 s | 7.1 s | 348 KB |
| Slow 3G (400 kbps)Phone, 390 px wide | 1.7 s | 8.1 s | 230 KB |
| Slow 3G (400 kbps)Computer, 1280 px wide | 2.4 s | 11 s | 348 KB |
“Needed Next” counts steps passed with the Next button because something sat on top of the click spot. Sizes use KB of 1024 bytes.
Limitations
- Each number is one run, measured with the app on the same laptop as the browser. It shows the effect of the slowed connection, not the spread of real networks, and there is no repeat to show how much it varies.
- The phone numbers come from a phone-sized window in a desktop browser, not from a real phone, so a real phone's own speed is not in them.
- 4 demos of 4 to 8 steps each, made from 2 sites. Other demos have other pictures and so other sizes.
- Sound was off for the timings. A voice adds its audio, which is listed under each demo and is not in the whole-demo size.
Outcomes from real viewers
There are no results from outside viewers yet; they appear here once outside viewers have used the demos.
Frequently asked questions
- No. They come from tests we ran ourselves: made-up data on a practice app, two public demos checked by machine, and three demos opened in a browser with a slowed network. Results from outside viewers have their own section and appear only when there are real ones.
- No. No native speaker has reviewed these yet. The language checks here are automatic and labelled that way.
- Each number is one run on a laptop, with the app on the same laptop and the network slowed by the browser. It shows what a slow connection does to a demo, not how much real networks vary.
- It means this build behaved this way on one practice app. It is not a rate for every app. Look at the covers on your own demo before you share it; every cover is a box you can turn off or add.