Patent US 9,681,202

The system and method disclosed herein relate to the establishment of both the location and identity of individual items in pictures or videos. Once the one or more items in the picture/video are identified and matched to images in a reference database, the locations of the items are established and additional information relating to the items is accessed. Collectively, the position data and additional data is merged in the electronic pictures or videos and then provided to a user as a merged data stream. Additional functionality related to those identified items can occur when those identified locations are “pointed to”, “clicked” or otherwise selected (e.g., purchase an item, request information, select another video stream, play a game, share the item, rate, “Like”, and the like).