Image search in walkthrough videos

1 . A method comprising:

generating a set of image sub-frames by 1) performing a cropping operation on each of a set of walkthrough videos of an environment to produce overlapping portions of frames of the walkthrough videos, and 2) performing a de-duplication operation to remove overlapping portions of frames that correspond to a same image search query, each of the set of image sub-frames corresponding to one of the walkthrough videos and associated with a timestamp;

generating an interface for display including a representation of a floorplan of the environment corresponding to the set of walkthrough videos and a camera path corresponding to at least one of the set of walkthrough videos, and including a histogram, the independent axis of the histogram comprising a timeline corresponding to the timestamps associated with the generated set of image sub-frames, the histogram including bars indicating both 1) instances of the generated set of image sub-frames within the set of walkthrough videos, and 2) a number of search results corresponding to the image search query; and

in response to a histogram bar being selected, populating a portion of the interface with walkthrough video search results corresponding to a time associated with the selected histogram bar.

2 . The method of claim 1 , wherein the image search query comprises a selection of a portion of a frame of a walkthrough video.

3 . The method of claim 1 , wherein the image search query comprises an image of an object or surface.

4 . The method of claim 1 , wherein the de-duplication operation removes a first overlapping portion of a frame that is within a threshold measure of similarity to a second overlapping portion of a frame.

5 . The method of claim 4 , wherein a difference between a feature vector corresponding to the first overlapping portion of the frame and a feature vector corresponding to the second overlapping portion of the frame is less than a threshold difference.

6 . The method of claim 1 , wherein the image search query corresponds to an object, and wherein marked locations within the representation of the floorplan comprises marking locations of the object within the floorplan based on the representation of the object within the walkthrough video.

7 . The method of claim 1 , wherein a marked location within the representation of the floorplan, when selected, causes the interface to be modified to provide for displaying a generated image sub-frame an identified portion of frame from the walkthrough video that corresponds to the marked location.

8 . The method of claim 1 , wherein each bar of the histogram corresponds to a walkthrough video of the set of walkthrough videos captured on a particular day.

9 . The method of claim 8 , wherein a height of each bar of the histogram is based on a number of generated image sub-frames included within the walkthrough video captured on the particular day corresponding to the bar of the histogram.

10 . The method of claim 1 , wherein each bar of the histogram, when selected, causes a floorplan corresponding to a walkthrough video captured at the timestamp associated with the selected bar to be displayed within the first interface portion.

11 . The method of claim 1 , wherein the de-duplication operation removes overlapping portions of frames that correspond to a same image search query based at least in part on a proximity of locations within the floorplan associated with each frame of the walkthrough video.

12 . The method of claim 1 , wherein the de-duplication operation removes overlapping portions of frames that correspond to a same image search query based at least in part on image features between the overlapping portions of frames of the walkthrough videos.

13 . The method of claim 1 , wherein the environment includes multiple floors, the floorplan spans the multiple floors, and wherein the first interface potion includes a scrollable representation of the multiple floors such that a viewer can scroll to view marked locations corresponding to the identified portions of frames across multiple floors.

14 . A system comprising:

a hardware processor; and

a non-transitory computer-readable storage medium storing executable instructions that, when executed by the hardware processor, cause the hardware processor to perform steps comprising:

generating a set of image sub-frames by 1) performing a cropping operation on each of a set of walkthrough videos of an environment to produce overlapping portions of frames of the walkthrough videos, and 2) performing a de-duplication operation to remove overlapping portions of frames that correspond to a same image search query, each of the set of image sub-frames corresponding to one of the walkthrough videos and associated with a timestamp

generating an interface for display including a representation of a floorplan of the environment corresponding to the set of walkthrough videos and a camera path corresponding to at least one of the set of walkthrough videos, the independent axis of the histogram comprising a timeline corresponding to the timestamps associated with the generated set of image sub-frames, the histogram including bars indicating both 1) instances of the generated set of image sub-frames within the set of walkthrough videos, and 2) a number of search results corresponding to the image search query; and

in response to a histogram bar being selected, populating a portion of the interface with walkthrough video search results corresponding to a time associated with the selected histogram bar.

15 . A non-transitory computer-readable storage medium storing executable instructions that, when executed by a hardware processor, cause the hardware processor to perform steps comprising:

generating a set of image sub-frames by 1) performing a cropping operation on each of a set of walkthrough videos of an environment to produce overlapping portions of frames of the walkthrough videos, and 2) performing a de-duplication operation to remove overlapping portions of frames that correspond to a same image search query, each of the set of image sub-frames corresponding to one of the walkthrough videos and associated with a timestamp

generating an interface for display including a representation of a floorplan of the environment corresponding to the set of walkthrough videos and a camera path corresponding to at least one of the set of walkthrough videos, and including a histogram, the independent axis of the histogram comprising a timeline corresponding to the timestamps associated with the generated set of image sub-frames, the histogram including bars indicating both 1) instances of the generated set of image sub-frames within the set of walkthrough videos, and 2) a number of search results corresponding to the image search query; and

in response to a histogram bar being selected, populating a portion of the interface with walkthrough video search results corresponding to a time associated with the selected histogram bar.