CONVERSATIONAL CALL QUALITY EVALUATOR

Aspects of the disclosure simulate a conversation over a real-time communication system and a reference script shared between the communication devices serves as a basis for comparison against the received speech-recognized conversation. A method of evaluating call quality of a real-time communication system that includes at least two communication devices is disclosed that includes receiving a reference script, the reference script containing linguistic contents of an audio signal being sent to one of the communication devices; generating an evaluation transcript by applying speech recognition to the audio signal being received; comparing the reference script with the evaluation transcript; and generating a call quality metric of the real-time communication system based on the comparison. The call quality metric may also include a communication delay which may be evaluated by determining a duration of a speaking turn; determining a duration of a listening turn from the audio signal being received by one of the communication devices; and estimating a communication delay of the audio signal based on the duration of the speaking turn and the listening turn.