QUERY RECOGNITION RESILIENCY DETERMINATION IN VIRTUAL AGENT SYSTEMS

A computer-implemented method includes: generating, by a computing device, a sample of a query, wherein the query maps to a predetermined categorization; processing the sample with a first natural language process, wherein the processing produces an output comprising a categorization of the sample; storing data regarding whether the categorization of the output matches the predetermined categorization; generating a report based on the stored data indicating a percentage of a plurality of samples of the query whose categorizations match the predetermined characterization; outputting the report; and receiving a second natural language process comprising updates to the first natural language process based on the report, wherein the updates increase the percentage of the plurality of samples that match the predetermined categorization when processed with the second natural language process relative to when the plurality of samples was processed with the first natural language process.