Distributed-memory multiprocessor computer system with directory-based cache coherency with ambiguous mappings of cached data to main-memory locations

A distributed-memory multiprocessor system uses fast and main coherency directories to implement cache coherency. The main directory is stored with user data in main memory and includes sufficient information to determine which memory cells have cached copies of user data stored in main memory. In addition, the main directories specify the states of the cached data. The fast directories cache only some of the main-directory information for only a fraction of the main-memory locations at any given time. The fast directories are tagless in one mode and use partial tags in another mode. The fast-directory information is accessed concurrently with main-directory information in response to data requests. Directory information is retrieved first from the fast directory and is used to launch predictive recalls. Subsequently received main-directory information is used to validate or invalidate the predictive recalls. If invalidated, determined recalls are issued, and memory access times are the same as they would have been without the fast directory. If validated, the predictive recalls reduce data access times. To the extent that the predictive recalls are successful, overall system performance is improved.