We asked four AIs for Sydney’s best mortgage broker. Twice, ChatGPT named nobody.
On 24 July 2026 we put two buying questions to four AI assistants, four times each. The strangest result wasn’t who got named. It was how often nobody did.
The questions were “best mortgage broker in Sydney” and a first-home-buyer version for Bondi. 32 answers, one sitting.
Ask an AI the same question four times and the answers shift. The band is that range, and a change only counts as movement once it clears it.The method in full →
Half a shortlist, half a shrug.
Asked “Best mortgage broker in Sydney – who do you recommend?” four times in one sitting, ChatGPT named nobody, then 17 businesses, then nobody again, then 11. The empty answers weren’t errors.
If your customers ask on the wrong try, no one gets recommended, including you. The physio study never returned an empty answer, so this trade behaves differently.
ChatGPT agreed just .14 – near Bondi.
0 = never the same answer · 1 = identical every time
The grey is the noise; the lume is the one reading the data will stand behind.
What we did
Two buying questions: “best mortgage broker in Sydney”, and a first-home-buyer version anchored to Bondi. Four assistants, four times each, one sitting. 32 answers, asked the way a customer would ask.
We count businesses, not strings, and named branches stay apart from parent brands. The methodology covers what counts as one.
97 businesses turned up across 32 answers, and half were named once. The grid below is what each assistant returned, ask by ask.
| assistant | names per answer | businesses named | named once | agreement | named in all four |
|---|---|---|---|---|---|
| ChatGPT | 0 · 17 · 0 · 11 | 21 | 14·67% | 0.22 / 0.33† | – twice named nobody |
| Perplexity | 5 · 5 · 5 · 3 | 8 | 3·38% | 0.44 | Shore Financial |
| Gemini | 8 · 12 · 11 · 11 | 17 | 6·35% | 0.54 | AFMS Group · Natloans · Shore Financial · Soren Financial · Villa Finance Group |
| Claude | 4 · 4 · 4 · 4 | 4 | 0·0% | 1.00 | Eden Emerald Mortgages · Get a Better Rate · Home Loan Experts · Soren Financial |
† ChatGPT twice named nobody; its score shows with and without those empty answers.
On the suburb-level question the lists were shorter, and ChatGPT held nothing at all across its four answers.
| assistant | names per answer | businesses named | named once | agreement | named in all four |
|---|---|---|---|---|---|
| ChatGPT | 6 · 5 · 8 · 8 | 20 | 15·75% | 0.14 | none |
| Perplexity | 9 · 4 · 6 · 8 | 12 | 5·42% | 0.50 | Aussie Bondi Junction · Loan Market Unlimited · Mortgage Choice Bondi |
| Gemini | 1 · 5 · 4 · 8 | 10 | 6·60% | 0.34 | Bondi Broker |
| Claude | 4 · 4 · 4 · 4 | 5 | 1·20% | 0.80 | Borro · Brampton Finance · Eventus Financial |
Half were named once, never again
Same city, same questions, asked the same way. Mortgage broking returned more businesses than physiotherapy, and more of them named once.
Claude did the opposite: the same four names, four times out of four. It was the steadiest and the narrowest answer in the study.
Set the two trades side by side and the churn is worse where the money is bigger.
| same city, same weeks | businesses named | named once | held all four |
|---|---|---|---|
| Physiotherapists | 85 | 36 | 17 |
| Mortgage brokers | 97 | 50 | 17 |
Who held across four answers
Seventeen businesses were named every time an assistant answered its question, and we checked each against a real brokerage before printing it. Shore Financial and Soren Financial held for two different assistants each.
One name deserves a second look. Natloans is a Melbourne brokerage, named by Gemini in all four answers to a question about Sydney. Being recommended is not the same as being local.
None of this is a ranking. Whether a name appears or disappears says something about the assistant giving the answer, not about the broker.
Why this matters for measuring AI visibility
If two of four asks can return nobody at all, a single screenshot of an AI answer is worth even less than usual. A ranking published without a range, or without the empty answers counted, is noise dressed as signal.
This is why the methodology attaches a band and a date to every reading, and why The Southlume Index publishes rates with bands rather than bare ranks.
The assistant that names the most businesses agrees with itself the least.
Open questions
One study is one snapshot. But the same thing keeps showing up: the physio study found this shuffle too, in a different trade.
We’re calling that a tendency, not a rule, until many more industries are measured.
The Southlume Index is measuring these categories now.
Prompt checks are a sample; referral analytics are the ground truth of actual AI-driven visibility. A zero here is ‘not found on these prompts’, never ‘invisible’.