How do I update an AI agent after it is already live?

You can continue improving an AI agent after it has already been published.

Common updates include:

• Changing AI Instructions
• Adding or removing nodes
• Updating Router logic
• Replacing or adding knowledge
• Editing Articles
• Updating Data Files
• Changing Products
• Changing Calendar settings
• Adding or removing AI Answer skills
• Changing Human Handoff behavior

A safe update process is:

Change → Test → Publish → Verify

Do not assume that a small configuration change only affects one part of the workflow.

Edit the draft version first

When making changes, work on the editable version of the AI agent before publishing.

This lets you:

• Adjust instructions
• Change workflow logic
• Add new paths
• Update skills
• Test behavior

without immediately changing the experience customers are using.

Publish only when the changes are ready

Publishing makes the latest version available to the channels using that AI agent.

Before publishing, confirm that the new configuration is complete and tested.

Do not publish partial changes just because one part is ready.

What should I re-test after a change?

The amount of testing depends on what you changed.

For example:

If you changed AI Instructions

Re-test:

• Main customer questions
• Scope boundaries
• Skill triggers
• Human Handoff conditions
• Tone and response style

If you changed Router

Re-test:

• Every route
• Similar intents
• Ambiguous requests
• Fallback behavior

If you changed Document Search knowledge

Re-test:

• Questions covered by the updated source
• Similar wording
• Exceptions
• Questions where the answer should not be found

If you changed Data Search

Re-test:

• Known records
• Exact values
• Missing records
• Related datasets
• Lookup behavior

If you changed a skill

Re-test that skill from start to finish.

Updating AI Instructions

Changes to AI Instructions can affect more than wording.

For example, changing:

“Ask for location before providing a recommendation.”

to:

“Only ask for location when the recommendation depends on location.”

can change how many follow-up questions the agent asks.

After changing instructions, test realistic conversations rather than only reading the instruction text.

Updating Articles

Published Articles may be used by both customers and Document Search.

When an Article changes:

• Review the updated content
• Confirm the Article is published
• Test questions that depend on it
• Check that the new information is reflected in the AI response

If the Article changes an important business rule, test related workflow paths too.

Updating Documents and PDFs

If you replace or add written knowledge, check that:

• The correct file is available
• The updated source is ready
• Old conflicting content is no longer active
• Document Search uses the intended source

Then test the questions most likely to be affected.

Updating Data Files

When structured data changes, verify:

• New records are present
• Old records are removed when appropriate
• Column names are still correct
• Join keys still match
• Prices and values are current

Compare Data Search results with the source file.

Updating Products

When Products change, review:

• Product name
• Description
• Price
• Options
• Variant prices
• Photos
• Category
• Link
• Button label

Then test Product Card requests that should match the updated Product.

Updating Calendar

Calendar changes can immediately affect booking behavior after the new configuration is in use.

Review:

• Weekly hours
• Timezone
• Appointment length
• Gaps
• Time-slot capacity
• Minimum booking notice
• Maximum booking window
• Open for appointments
• Alerts and reminders

Then test AI Answer with Scheduling enabled against the updated availability.

Adding a new AI Answer skill

If you enable a new skill, do not only test the skill itself.

Also test whether AI Answer uses it at the right time.

For example, after enabling Scheduling, test:

• Booking requests
• Non-booking questions
• Unavailable times
• Missing customer information
• Normal questions that should not trigger Scheduling

Removing a skill

If you remove a skill, test what happens when a customer asks for that capability.

For example, if Scheduling is removed, decide whether the AI agent should:

• Explain that booking is not available
• Send the customer elsewhere
• Use Human Handoff
• Continue with another workflow

Do not leave the old behavior implied in AI Instructions.

Updating Router logic

Routing changes can affect many paths.

After changing Router:

1. Test each intended route
2. Test phrases that could match more than one route
3. Test the fallback path
4. Test Human Handoff
5. Test multi-intent messages

A routing change can make a previously correct path unreachable.

Adding a new workflow path

When adding a new path, test both:

• The new path
• Existing paths that are similar

For example, adding a new Product Inquiry route may affect an existing Pricing route if the intents overlap.

Removing a node or path

Before removing something, check whether:

• Another path depends on it
• Router still points to it
• AI Instructions mention it
• Human Handoff expects information from it
• Later nodes use values collected there

Removing one node can affect the rest of the workflow.

Updating Human Handoff

If you change handoff behavior, test:

• Customer explicitly asking for a person
• Handoff after structured intake
• Handoff after failed AI resolution
• Conversation context passed to the team
• What the customer sees before and after handoff

Make sure the AI agent does not continue when a person is supposed to take over.

Test with a fresh conversation

After important changes, start a new conversation.

Old conversation history can hide problems because it may already contain:

• Collected values
• Prior routing decisions
• Earlier context
• Previous answers

A fresh conversation shows how the updated workflow behaves from the beginning.

Test the paths most affected first

You do not always need to re-test every possible conversation after a small change.

Start with:

• The path you changed
• Closely related paths
• Any shared Router logic
• Any shared knowledge or skills

Then run your main regression tests.

Keep a small regression test set

For important AI agents, maintain a reusable set of test conversations.

For example:

1. Main FAQ question
2. Main Document Search question
3. Main Data Search lookup
4. Main calculation
5. Main booking request
6. Main form
7. Human Handoff request
8. Out-of-scope question

Run these after major updates.

Verify the published version

After publishing:

• Start a fresh conversation
• Test the main path
• Test the changed feature
• Confirm the latest content appears
• Confirm old behavior is no longer present

Do not assume publishing worked just because the configuration looked correct in the builder.

Check the actual customer channel

If customers use:

• Web messenger
• WhatsApp
• Chat Page

test the important change in the real channel where possible.

Some experiences, such as cards, images, booking, or handoff, may look different outside the builder.

Avoid changing too many things at once

Large updates are harder to troubleshoot.

When possible, separate major changes.

For example:

1. Update knowledge
2. Test
3. Update routing
4. Test
5. Add a new skill
6. Test

This makes it easier to identify what caused a problem.

Document important changes

For business-critical AI agents, keep a simple record of major updates.

Examples:

• New pricing source added
• Scheduling enabled
• Cancellation policy updated
• New Router path added
• Handoff rule changed

This can help your team understand why behavior changed later.

What changes require the most caution?

Pay extra attention when changing:

• Pricing
• Calculations
• Booking rules
• Human Handoff
• Policies
• Router logic
• Structured data
• Business-critical instructions

These changes can affect customer outcomes directly.

A safe update process

Use this sequence:

1. Make the change

Update the draft configuration, knowledge, Product, Calendar, or workflow.

2. Test the affected path

Confirm the change behaves as expected.

3. Run key regression tests

Make sure the change did not break important existing paths.

4. Publish

Publish only when the updated configuration is ready.

5. Verify

Start a fresh conversation and confirm the published behavior.

Example: changing a cancellation policy

Suppose the cancellation policy changes from 24 hours to 48 hours.

A safe update would be:

1. Update the Article or document
2. Remove any old conflicting source
3. Test Document Search
4. Check AI Instructions for references to 24 hours
5. Publish the updated workflow if needed
6. Start a fresh conversation
7. Ask several cancellation questions
8. Confirm the AI now uses the 48-hour rule

Example: adding Scheduling

A service business wants customers to book appointments.

A safe update would be:

1. Create and configure Calendar
2. Enable Scheduling in AI Answer
3. Connect the correct Calendar
4. Update AI Instructions
5. Test normal booking
6. Test unavailable times
7. Test missing contact information
8. Test non-booking questions
9. Publish
10. Verify from a fresh customer conversation

Example: adding a new Product catalog

A business adds Product Cards.

A safe update would be:

1. Create or import Products
2. Review names, prices, options, and links
3. Enable Product Cards
4. Define when Product Cards should appear
5. Test specific Product requests
6. Test broader recommendation requests
7. Test unrelated questions
8. Publish
9. Verify the live experience

Tips for updating a live AI agent

• Make changes in the editable version first.
• Re-test the part you changed.
• Re-test closely related paths.
• Use a small regression test set.
• Start fresh conversations after major changes.
• Publish only complete, tested changes.
• Verify the published behavior afterward.
• Test in the real customer channel when possible.
• Avoid changing several critical systems at once.
• Keep important knowledge, routing, and skill behavior aligned.

Related guides

For more information, see:

• How should I test my AI agent before publishing?
• How do I plan an AI agent?
• How should I organize my knowledge in AskHandle?
• Why isn't my AI agent finding the right answer?
• How should I write AI Instructions?