Only testing with standardized questions written by developers often fails to reveal the expression problems that will be encountered after going online. Customers may omit the subject, write the wrong product name, or ask two questions in one sentence. The test question set should cover these expressions, as well as requests that should not be answered in the first place. Unanswered questions are equally important, as they check whether the system will acknowledge insufficient data.

First clarify the questions and conditions

Write the applicable conditions, reference basis and acceptable handling methods for each question. Some questions allow several expressions, and the focus of evaluation is on whether the facts and the next step are correct, without word-for-word comparison. Tests involving account information use desensitized samples and check identity and permission boundaries. The question sets are grouped by business modules, making it easier to find the parts that need to be retested after updating your knowledge.

Put the rules into the actual process

For example, if you ask where the order is, you can set three conditions: logged in and having an order, identity not yet verified, and order canceled. Correct responses should not be exactly the same. Then add functional problems that are assumed not to exist, and check whether the system has programmed entrances and exits; when the customer requests to switch to labor, check whether the corresponding process will be entered immediately.

Check the processing results item by item

  • Every time you modify the search or prompt rules, save the results under the same question set and record the mistakes.
  • Manual review focuses on missing premises, unsupported sources, and operational risks rather than just giving an overall score.
  • New and real questions will be added to the question set after desensitization, and old questions will also be updated as the rules change.
  • Isolating critical errors helps determine which issues must be resolved before release.

How to review before and after going online

Someone must be responsible for updating the standard basis in the test set. After the old rules are cancelled, the system will no longer be required to answer the old conclusions. By entering conditions when saving failed questions, the team can reproduce the problem instead of just getting a vague statistics on the number of errors.