Ask AI vendors a bounded question that reveals task fit, evidence, account limits and failure behaviour during a short product demonstration.

Direct answer: Ask one question that names your task, a difficult input, the evidence you need, the account being demonstrated and the failure you cannot accept. Request an observable answer or a documented follow-up, not a promise that the tool "can handle" the situation. If the presenter cannot disclose the setup or show how the result is checked, treat the answer as an unresolved sales claim.

Public product demonstrations reward broad questions with broad answers. "Is it secure?" or "Can it work for small businesses?" lets a speaker choose the definition, context and success case. A useful question narrows the room available for substitution.

London affects the question because reaching an evening meetup or conference can consume more time than the advertised session. Use the scarce question period to resolve an adoption-stopping unknown, not to repeat a feature list.

This does not make a stage demonstration equivalent to an independent test. It helps you leave a busy London session with one fact or one clearly labelled unknown that can inform the focused workflow described in A Useful AI Workflow for Focused Knowledge Work.

Applies to: live or streamed demonstrations where audience questions are limited. The presenter may not be authorised to answer contractual, legal or security questions, so request the responsible document or contact rather than forcing an improvised answer.

Use the observable-answer question

The observable-answer question is a five-part editorial method:

  1. Task: the job you need completed.
  2. Difficult input: the condition most likely to expose a limitation.
  3. Evidence: what lets a reviewer verify the result.
  4. Configuration: the application, account tier, model or connected source in use.
  5. Failure boundary: the error that makes the result unusable.

Put those parts into one sentence. For example: "For a 30-page report containing two conflicting dates, can this account identify the conflict, return both page references and mark uncertainty rather than choosing one date, and what happens if the PDF text cannot be extracted?"

That question is stronger than asking whether the product reads PDFs. It identifies a task, a difficult case, visible evidence, the demonstrated account and a reject condition. The presenter can show it, point to documentation, or say it has not been tested.

Prepare before entering the room

Write your question before the demonstration. Start from a real decision you face, but remove client, employee and confidential details. Do not upload a live file to a public demo account simply because a presenter invites examples.

Bring a short synthetic example only if the event permits it and the presenter explains where it will be processed or retained. The ICO says data minimisation means collecting or using only what is necessary for the stated purpose. That principle is especially relevant when an event creates pressure to share more context than a demonstration needs. ICO guide to data-protection principles

Check the published agenda and product documentation first. Do not spend scarce question time asking for a feature list already on the website. Mark which facts are current, which apply only to a named tier and which remain unclear.

Ask for the setup, not just the output

An output on a screen is incomplete evidence. Ask which application is being used, which model or mode is selected, whether files or knowledge sources were loaded earlier, whether an integration supplied information and whether a person edited the result before it appeared.

This is not an accusation of deception. Demonstrations are compressed. The presenter may legitimately use a prepared workspace. Your task is to avoid carrying a result into a different configuration and expecting identical behaviour.

The UK Government AI Playbook advises its government audience to understand AI limitations, keep meaningful human control and manage the full lifecycle. It also warns that AI output is not guaranteed accurate. The institutional scope matters, but the practical lesson applies to a demo question: ask how the output becomes reviewable, not merely fluent. AI Playbook for the UK Government

Make the evidence concrete

Good evidence depends on the task. For document work, request page or section references. For a calculation, request inputs, units and a reproducible formula. For classification, request the label definition, uncertain cases and a confusion table rather than a single accuracy percentage. For an action-taking system, request a preview, approval point, activity record and reversal route.

Do not ask a prompt to guarantee accuracy. Ask what the human must compare with the original and what the application does when evidence is absent. NIST's voluntary AI Risk Management Framework organises risk work around govern, map, measure and manage, and treats testing and evaluation as lifecycle activities. A stage answer is one observation within that work, not proof of trustworthiness. NIST AI RMF resources

Fit the question to the available time

Suppose a London demo has an illustrative 12-minute question period. Eight people want to speak. If introductions and broad framing take 45 seconds per person, they consume 8 × 45 seconds = 360 seconds, or 6 minutes. Only 6 minutes remain for answers.

Your prepared question takes 25 seconds to ask. A focused answer and one clarification take an illustrative 2 minutes 35 seconds. The complete exchange is 3 minutes. After the eight introductions above, the remaining 6 minutes support only 6 minutes ÷ 3 minutes = 2 such exchanges. Four exchanges is the theoretical maximum only if the entire 12 minutes is available and introductions create no separate overhead: 12 ÷ 3 = 4.

The consequence is practical. Do not combine privacy, pricing, integrations, accuracy and roadmap into one speech. Choose the unknown that could stop your evaluation. Submit the others through the documented follow-up route. These timings are planning assumptions, not observations from a particular event.

Recognise evasive and useful answers

A useful answer defines scope. It may say that the feature works only on a particular plan, that a cited-source mode is required, or that the presenter needs to confirm retention terms later. A limitation can be more valuable than an unqualified yes.

An evasive answer changes the task, cites an unnamed customer, relies on a future roadmap, or replaces a measurable condition with an adjective. Respond once: "Which part can you show or document for the account on screen?" If the answer remains general, record it as unverified and let another attendee ask a question.

Do not debate security architecture, law or contractual interpretation with a presenter who does not own those subjects. Ask for the current security documentation, data-processing terms, accessibility statement or pricing page and the date or version that applies. Vendor material establishes what the vendor claims, not independent real-world performance.

Record the answer without overstating it

Write four lines immediately after the London session: your question, the configuration named, what was shown or documented, and what remains unknown. Distinguish the presenter's statement from your inference. If you photograph a slide or record audio, follow the event rules and obtain permission where required.

Do not write "the tool passed" when the presenter showed one curated example. A fair note might say: "The presenter demonstrated page-linked answers on the enterprise workspace using the supplied sample. Behaviour on our account and difficult PDFs remains untested."

UK copyright, confidentiality, recording and data-protection requirements depend on the material, event terms and intended reuse. Access to a presentation does not automatically authorise republication. Seek qualified UK advice when your proposed use has legal or contractual consequences.

What to do in the next 24 hours

Before the event, spend 15 minutes writing one observable-answer question and one acceptable follow-up. During the session, note the exact setup and whether the response was shown, documented or promised. Within 24 hours, open the cited documentation and preserve the relevant date, plan and limitation.

Then decide whether the evidence justifies a small independent test. Stop if the required account, data terms or failure behaviour remains unclear. Do not book a sales call merely to convert a vague stage answer into another vague conversation. A call is warranted only when you have a bounded unanswered requirement and the right owner can answer it.

Frequently asked questions

What is the single best question to ask an AI vendor?

There is no universal best question. Ask about the failure that would stop your own task. Name the input, expected output, required evidence, demonstrated configuration and unacceptable error. For document analysis, that might concern missing pages; for customer-facing writing, unsupported commitments; for automation, an unauthorised action. A generic security or accuracy question usually produces a generic answer. The exception is an introductory event where you genuinely have no defined task. In that case, listen first and use the session to decide whether further evaluation deserves time.

Should I ask about accuracy percentages?

Ask only if the presenter can define the task, test set, metric, product version and conditions behind the percentage. An aggregate score may hide the exact failure you care about. Request examples of errors and how uncertain cases are handled. If the vendor cannot share the underlying method, record the number as a vendor claim rather than independent evidence. A percentage can still help compare like-for-like evaluations, but it should not replace a test on representative material or the human review required for consequential work.

Is it rude to ask the presenter to show a failure case?

No, if the request is concise and relevant. A useful product discussion includes limits and recovery. Phrase it as a task requirement: "What does the user see when the source is unreadable?" rather than challenging the speaker's competence. The presenter may not have time or permission to run an unplanned case. Accept a documented follow-up when it identifies the owner and source. If the event is explicitly a high-level talk rather than a product evaluation, save detailed testing questions for the appropriate session.

Can I give the presenter one of my real work files?

Usually use a synthetic substitute first. A public demo does not establish your employer's permission, the provider's data terms, the account owner, retention or who can see the screen. Remove personal and confidential information, but do not call a file anonymous merely because names were replaced. Structural details may still identify people or clients. If a real file is genuinely necessary, obtain organisational approval and verify the applicable terms before the event. Regulated, privileged or sensitive material may require specialist advice.

What if the presenter promises a feature is coming soon?

Treat it as a future claim, not a current capability. Ask whether there is a public release note or dated roadmap, and make your present decision using the product available now. Do not buy a plan or design a workflow that fails without an unreleased feature unless the commercial agreement provides an acceptable, enforceable commitment and you have reviewed it appropriately. Even then, maintain a fallback. Announcements, private previews and general availability are different states, and access may vary by account, region or administrator setting.

Should I accept a sales call as the follow-up?

Accept one only when the unresolved question matters enough and the call will include someone authorised to answer it. Send the bounded question in advance, request relevant documents and set a short agenda. Decline a discovery call when you only need public pricing, terms or a technical limit that should be documented. A call costs staff time and may add pressure without improving evidence. If the vendor refuses to answer a material requirement without an open-ended sales process, include that friction in your product evaluation.

Sources and verification

Twokq Tech

This article is practical guidance. Apply it in proportion to your tools, evidence, risks, and responsibilities.