INVALIDATING TRANSLATION LOOKASIDE BUFFER ENTRIES IN A VIRTUAL MACHINE (VM) SYSTEM

One embodiment of the present invention is a technique to invalidate entries in a translation lookaside buffer (TLB). A TLB in a processor has a plurality of TLB entries. Each TLB entry is associated with a virtual machine extension (VMX) tag word indicating if the associated TLB entry is invalidated according to a processor mode when an invalidation operation is performed. The processor mode is one of execution in a virtual machine (VM) and execution not in a virtual machine. The invalidation operation belongs to a non-empty set of invalidation operations composed of a union of (1) a possibly empty set of operations that invalidate a variable number of TLB entries, (2) a possibly empty set of operations that invalidate exactly one TLB entry, (3) a possibly empty set of operations that invalidate the plurality of TLB entries, (4) a possibly empty set of operations that enable and disable use of virtual memory, and (5) a possibly empty set of operations that configure physical address size, page size or other virtual memory system behavior in a manner that changes the manner in which a physical machine interprets the TLB entries.