Indoor semantic map updating method and system based on semantic information extraction

1. An indoor semantic map updating method based on semantic information extraction, comprising:

issuing a crowdsourcing task to all mobile terminals, wherein the crowdsourcing task comprises a position acquisition task and a short video collection task, the position acquisition task is to acquire a geographic position where the mobile terminal is located according to GPS positioning, and the short video collection task is to photograph an indoor object containing rich semantics;

waiting for any mobile terminal to execute the crowdsourcing task, and receiving a task result thereof;

preprocessing a short video collected by the short video collection task, and extracting a common key frame sequence, which specifically comprises: extracting all key frames from the short video, and including all key frames into a plurality of key frame sequences; and extracting a common part from the plurality of key frame sequences to obtain the common key frame sequence;

extracting an accurate text sequence from the common key frame sequence, which specifically comprises: recognizing text from the common key frame sequence, and including all recognized text into a plurality of text sequences; and extracting the accurate text sequence from the plurality of text sequences by using a Markov random field; and

updating an indoor semantic map according to the common key frame sequence and the accurate text sequence, which specifically comprises: extracting an image feature from the common key frame sequence; extracting an unchanged text score and an unchanged text set from the accurate text sequence; separately calculating a room contour energy term, an unchanged text energy term and an unchanged neighbor text energy term according to the image feature, the unchanged text score, the unchanged text set and a preset weight; calculating a total energy term according to the room contour energy term, the unchanged text energy term and the unchanged neighbor text energy term; and positioning the accurate text sequence onto the indoor semantic map by using the total energy term, and updating the indoor semantic map.

2. The method of claim 1 , wherein extracting all key frames from the short video comprises: removing all images having similar heights with other images from the short video by using a key frame algorithm, the remaining images being deemed as the key frames.

3. The method of claim 1 , wherein extracting a common part from the plurality of key frame sequences to obtain the common key frame sequence comprises:

specifying one of the plurality of key frame sequences as a seed sequence;

additionally selecting an unlabeled sequence from the plurality of key frame sequences, and calculating a sequence distance between the seed sequence and the additionally selected sequence and a length difference of the two sequences;

calculating a longest common sub-sequence of the seed sequence and the additionally selected sequence according to the sequence distance between the seed sequence and the additionally selected sequence and the length difference of the two sequences;

calculating a similarity score of the two sequences according to the longest common sub-sequence of the seed sequence and the additionally selected sequence, and determining whether the two sequences are similar according to the similarity score;

judging whether the length of the longest common sub-sequence of the seed sequence and the additionally selected sequence reaches more than half of the length of the additionally selected sequence, if so, marking the additionally selected sequence, and otherwise, not marking the additionally selected sequence; and

reselecting another unlabeled sequence from the plurality of key frame sequences, and sequentially executing the above operations until all sequences in the plurality of key frame sequences are labeled.

4. The method of claim 1 , wherein extracting the accurate text sequence from the plurality of text sequences by using a Markov random field comprises:

obtaining each hidden state node and an observation node corresponding thereto in the plurality of text sequences;

obtaining a probability function between every two hidden state neighbor nodes and the probability function between each hidden state node and the observation node corresponding thereto according to each hidden state node and the observation node corresponding thereto in the plurality of text sequences;

obtaining a joint probability of the plurality of text sequences according to the probability function between every two hidden state neighbor nodes and the probability function between each hidden state node and the observation node corresponding thereto;

obtaining optimal estimation of any node by using a maximum likelihood estimation method on the joint probability of the plurality of text sequences;

obtaining information between any two nodes according to the optimal estimation and reliability of the node; and

extracting the accurate text sequence according to the information between any two nodes.

5. The method of claim 1 , wherein positioning the accurate text sequence onto the indoor semantic map by using the total energy term comprises:

specifying each text sequence successively;

calculating a sequence distance between the specified text sequence and an indoor semantic map overall sequence and a length difference of the two sequences;

calculating a longest common sub-sequence of the specified text sequence and the indoor semantic map overall sequence according to the sequence distance between the specified text sequence and the indoor semantic map overall sequence and the length difference of the two sequences;

calculating a similarity score of the specified text sequence and the indoor semantic map overall sequence according to the longest common sub-sequence of the two sequences; and

after each text sequence is traversed, positioning the text sequence having the highest similarity score onto the indoor semantic map.

6. An indoor semantic map updating system based on semantic information extraction, wherein the indoor semantic map updating method of claim 1 is used.

7. An indoor semantic map updating system based on semantic information extraction, wherein the indoor semantic map updating method of claim 2 is used.

8. An indoor semantic map updating system based on semantic information extraction, wherein the indoor semantic map updating method of claim 3 is used.

9. An indoor semantic map updating system based on semantic information extraction, wherein the indoor semantic map updating method of claim 4 is used.

10. An indoor semantic map updating system based on semantic information extraction, wherein the indoor semantic map updating method of claim 5 is used.