Run a Manual ChatGPT Visibility Audit

Prerequisites: Lectures 1, 2, 3, 4, 6, 8, 9, 10 and 11.

Before this lecture, you should be able to read a ChatGPT answer as assembled evidence, split it into the four rooms, recognise category drift, and make one property more consistent across name, address and pages. You should also be able to weigh owned pages, directories, reviews, map listings, citeable facts, location signals, review context and stale evidence. Now we turn those reading habits into a repeatable check.

On a rainy Tuesday in a teaching exercise, I put three ChatGPT answers about the same small hotel on one table. One called the place a “boutique hotel near the old town.” One called it a “guesthouse in the hills above the town.” One said it was “suitable for guests arriving by train,” which sounded pleasant until the owner pointed out the steep road from the station. None of the answers was ridiculous. That was the irritating part.

This is where many owners become impatient. They want to correct the model, or rewrite the whole homepage, or ask ten more questions until one answer feels better. I understand the impulse. But if you move too quickly, the answer turns into a loud mosquito in the room: annoying, mobile, impossible to study. A manual audit pins it down gently. Date the answer, mark its rooms, check its source clues, and decide which public evidence deserves repair.

Begin with a clean, boring prompt

The first audit prompt should not flatter the property or accuse ChatGPT of being wrong. Ask a plain guest-style question. For example: “How would you describe [canonical property name] in [town] to a guest considering a stay?” If the property has a common name, include the town and region. If the name is unusual, still include the town, because the place room should be tested deliberately.

A manual ChatGPT visibility audit is a dated reading of one property through repeated plain prompts, because a single polished answer cannot show which evidence failed. That definition is deliberately dry. The work is dry. Good audit practice feels closer to checking room keys than writing advertising copy.

Do not start with “Use only official sources” or “Do not hallucinate.” Those instructions may be useful in other work, but they can hide the ordinary answer a guest might receive. In this course we want to see the answer as it naturally arrives: the neat paragraph, the confident category, the smooth promise room, the missing edge about parking or seasonality. The audit begins with observation before repair.

A clean first prompt also prevents the owner from steering the answer too hard. If you write, “Tell me why our family-run guesthouse is not a luxury resort,” you are no longer auditing the public picture. You are coaching the answer. Later you can test corrections with narrower prompts. First, let the model show its default shape.

I usually ask for one compact answer, not a long report. A short answer is easier to mark. It also resembles what a guest might read while comparing properties quickly. Long answers create more phrases, more guesses and more places for the audit to drown.

Turn one answer into audit marks

Once the answer appears, copy it exactly. Do not rewrite it while copying. Do not fix the tone. Keep even the awkward phrase, because the awkward phrase may show the retrieval path or inference that matters.

Then mark the answer by the four rooms. The identity room asks whether ChatGPT named the right property and connected it to one consistent entity. The category room asks whether the property type is right: hotel, guesthouse, B&B, inn, agriturismo or another stay type. The place room asks whether the location, address, town, landmark or area is accurate enough for a guest. The promise room holds claims about amenities, atmosphere, service, value and suitability.

This marking should be humble. A sentence can be correct in one room and weak in another. “A family-run boutique hotel near the historic centre with quiet rooms and easy access by train” may carry several different statuses at once. “Family-run” might be supported. “Boutique hotel” may be category drift. “Near the historic centre” may depend on how guests walk. “Easy access by train” may ignore the steep road. “Quiet rooms” may be repeated in reviews, though not guaranteed for every room.

After the four rooms, mark the evidence status. I use four simple labels in my own notes: supported, partly supported, unsupported, and check source. These are working labels, not formal metrics. Supported means you can find stable public evidence. Partly supported means the answer is close but missing a condition. Unsupported means the claim cannot be traced to stable public evidence. Check source means the phrase smells as if it came from a directory, review profile, map listing or old page, but you have not located it yet.

An unsupported claim is a factual-sounding AI statement that cannot be traced to stable public evidence. It may be pleasant. It may even be plausible. Still, for visibility work, it is unusable until you can connect it to a citeable fact or remove the pressure that produced it.

Use a small prompt routine, not a prompt storm

A manual prompt routine is a repeatable set of plain prompts for checking AI identity, category, place and claims. Repeatable is the important word. If you invent new prompts each time, you cannot tell whether the property picture changed or your question changed.

For a small Italian hotel or guesthouse, five prompts are usually enough for one audit round. First, ask for the general description. Then ask, “What kind of property is [name] in [town]?” That tests the category room. Next ask, “Where is [name] located, and what area or landmarks is it associated with?” That tests the place room. Then ask, “What services or amenities does [name] appear to offer?” That opens the promise room. Finally ask, “Which parts of this description seem uncertain or would need checking?” This last prompt does not prove anything, but it may reveal where the answer is thin.

Notice the wording: “appears to offer” and “would need checking.” I do not ask the model to certify truth. ChatGPT is not your registry clerk, and it is not standing in the breakfast room counting tables. The routine invites uncertainty into the answer so you can inspect it.

There is a small trap here. If you ask ChatGPT to list its sources, it may give source-like language that sounds stronger than the evidence deserves. Treat any named or implied source as a clue, not as proof. If it says a claim may come from booking listings or reviews, you still need to check your public profiles. If it names your own location page, open the page and read the sentence. The audit happens in your eyes, not in the model’s apology voice.

Run the routine in one language at a time if your property receives questions in more than one language, but keep the comparison simple. Do not introduce a second language before you have finished marking the first set. Mixing prompts, languages and corrections in the same sitting makes the audit notebook look like a plate of tangled tagliatelle.

Keep an answer log that a tired manager can read

An answer log is a record of prompts, dates, answers, suspected sources and repair decisions. It does not need software. A document or spreadsheet is enough. The test is whether another staff member could open it after lunch and understand what happened.

Each entry should capture the date, the exact prompt, the exact answer or relevant excerpt, the four-room marks, suspected source trail, and next repair decision. If the answer says “private parking available” and parking is reservation-only, write that down as a promise-room issue. Then note where the current parking rule appears: location page, booking listing, map profile, directory fields. If the official page is vague, the repair target is probably your owned page first. If the page is clear but a directory field is wrong, the next action is different.

The answer log protects you from overreacting. One odd answer can happen. A repeated wrong category across several prompts deserves attention. A single invented amenity may be a loose inference. The same invented amenity appearing after different guest-style prompts may mean old public wording still has oxygen. Without a log, owners remember the most annoying sentence and forget the pattern.

In a composite scenario, a small hill-town guesthouse ran three audit rounds over a season. The first round called it “ideal for car-free travellers.” The owner disliked the phrase but did not change anything immediately. In the second round, the answer repeated “walkable from the station.” In the third, after a review mentioned a difficult climb, ChatGPT softened the wording to “reachable from the station, though guests may prefer a taxi with luggage.” That final phrasing was closer to reality. The useful detail was not that ChatGPT became perfect. It was that the log showed the place room moving toward a more honest edge.

The log should also record what you decide not to fix. Some phrases are harmless atmosphere. Some platform categories cannot be changed. Some reviews will stay rough because guests wrote them honestly. Maintenance becomes calmer when the log distinguishes “repair now,” “watch,” and “ignore unless repeated.”

Match the problem to the smallest evidence repair

A manual audit is not a demand to rewrite every public page. It is a sorting table. Wrong name? Check the canonical property name, NAP consistency and the highest-control pages. Wrong category? Strengthen category evidence on the About page and structured profiles. Weak place room? Look at the address, landmark evidence and arrival wording. Swollen promise room? Turn repeated praise or vague service language into citeable facts with conditions.

The smallest repair is often a sentence. If ChatGPT says “spa-style retreat” because old text and reviews repeat “relaxing,” you may not need a new website section. You may need one clear service sentence: “The property offers quiet rooms and breakfast, but it does not operate a spa or wellness centre.” If the answer says “central Verona” when the property is in a nearby village, the location page may need a sharper line explaining the village, route and distance without pretending to be in the city.

Sometimes the best repair is outside the website. A map listing may hold the wrong category. A directory may carry an old address. A booking description may copy a service promise from before renovation. Still, owned pages matter because they let you publish the current version in calm wording. When outside evidence is slow to change, your own page should be the clearest current source in the trail.

Be careful with unsupported claims that sound flattering. “Romantic,” “exclusive,” “luxury,” “hidden gem,” “perfect for families,” and “easy without a car” can all damage guest expectations when the evidence is thin. The audit question is not whether you like the phrase. The question is whether a careful public reader could support it from stable evidence.

After repair, run the same prompt routine again later in a separate entry. Do not change five things and then ask a completely different prompt. You will not know what happened. Keep the routine boring enough that small evidence changes become visible.

The audit changes the owner’s posture

Before this lecture, much of the course asked you to read parts of an answer: source trail, category, place, reviews, stale evidence. The audit brings those habits into one sitting. It also changes the emotional posture. The owner stops asking, “Why did ChatGPT say this about me?” and starts asking, “Which public evidence made this sentence easy to say?”

That shift matters because small hospitality businesses live inside public fragments. A family name on one page. A road description on another. A booking field. A guest phrase about breakfast. A map category. A seasonal service line that should have been removed. ChatGPT does not experience the property. It assembles a readable account from traces, patterns and sometimes inference.

A good audit does not chase every strange answer. It teaches the property to leave clearer traces. The routine is manual because manual reading is still the best way to notice when a phrase is correct but misleading, attractive but unsupported, or current in Italian staff memory and stale in public wording. Tools can help later. First, the eye must learn the room.

What to remember

A manual prompt routine is a repeatable set of plain prompts for checking AI identity, category, place and claims. Keep it stable enough that changes in answers mean something.

An answer log is a record of prompts, dates, answers, suspected sources and repair decisions. It protects you from panic after one strange answer.

An unsupported claim is a factual-sounding AI statement that cannot be traced to stable public evidence. Pleasant unsupported claims are still weak visibility assets.

Four rooms of Italian hospitality visibility are the name room, the category room, the place room and the promise room, because ChatGPT must recognise who the property is, what kind of property it is, where it belongs and which promises public evidence can support.

A good audit ends with the smallest useful evidence repair: one clearer page sentence, one corrected listing, one better location line, or one dated service clarification.