Augmented reality scene image processing method and apparatus, electronic device and storage medium

1. An Augmented Reality (AR) scene image processing method, comprising:

acquiring shooting pose data of an AR device;

acquiring presentation special effect information of a virtual object corresponding to the shooting pose data in a reality scene based on the shooting pose data and pose data of the virtual object in a three-dimensional scene model configured to represent the reality scene; and

displaying an AR scene image through the AR device based on the presentation special effect information,

wherein the three-dimensional scene model is generated in the following manner:

acquiring multiple reality scene images corresponding to the reality scene; and

generating the three-dimensional scene model based on the multiple reality scene images;

wherein generating the three-dimensional scene model based on the multiple reality scene images comprises:

extracting multiple feature points from each reality scene image of the multiple reality scene images; and

generating the three-dimensional scene model based on the multiple feature points and a pre-stored three-dimensional sample image matched with the reality scene, wherein the pre-stored three-dimensional sample image comprises a pre-stored three-dimensional image representing a morphology feature of the reality scene.

2. The method of claim 1 , wherein acquiring the presentation special effect information of the virtual object corresponding to the shooting pose data in the reality scene based on the shooting pose data and the pose data of the virtual object in the three-dimensional scene model configured to represent the reality scene comprises:

acquiring the presentation special effect information of the virtual object corresponding to the shooting pose data based on the shooting pose data, the pose data of the virtual object in the three-dimensional scene model, and the three-dimensional scene model.

3. The method of claim 1 , wherein acquiring the shooting pose data of the AR device comprises:

acquiring a reality scene image shot by the AR device; and

determining shooting pose data corresponding to the reality scene image based on the reality scene image and a pre-stored first neural network model for positioning, wherein the shooting pose data corresponding to the reality scene image comprises at least one of shooting position information or shooting orientation information.

4. The method of claim 3 , wherein the pre-stored first neural network model is trained according to the following step:

training the pre-stored first neural network model based on multiple sample images obtained by shooting of the reality scene in advance and shooting pose data corresponding to each of the multiple sample images.

5. The method of claim 1 , wherein acquiring the shooting pose data of the AR device comprises:

acquiring a reality scene image shot by the AR device; and

determining shooting pose data corresponding to the reality scene image based on the reality scene image and an aligned three-dimensional sample image, wherein the shooting pose data corresponding to the reality scene image comprises at least one of shooting position information or shooting orientation information, and the aligned three-dimensional sample image is a three-dimensional sample image obtained after feature point alignment of a sample image library obtained by shooting of the reality scene in advance and the pre-stored three-dimensional sample image.

6. The method of claim 5 , wherein determining the shooting pose data corresponding to the reality scene image based on the reality scene image and the aligned three-dimensional sample image comprises:

determining a feature point, matched with a feature point in the reality scene image, in the three-dimensional sample image based on the aligned three-dimensional sample image;

determining a target sample image matched with the reality scene image in the sample image library based on coordinate information of the feature point in the three-dimensional sample image in the aligned three-dimensional sample image, wherein the sample image library comprises multiple sample images obtained by shooting of the reality scene in advance and shooting pose data corresponding to each of the multiple sample images; and

determining the shooting pose data corresponding to the target sample image as the shooting pose data corresponding to the reality scene image.

7. The method of claim 1 , wherein after acquiring the shooting pose data of the AR device, the method further comprises:

acquiring a reality scene image shot by the AR device; and

determining attribute information corresponding to the reality scene image based on the reality scene image and a pre-stored second neural network model that is configured to determine the attribute information corresponding to the reality scene image, wherein

acquiring the presentation special effect information of the virtual object corresponding to the shooting pose data in the reality scene based on the shooting pose data and the pose data of the virtual object in the three-dimensional scene model configured to represent the reality scene comprises:

acquiring the presentation special effect information of the virtual object corresponding to the shooting pose data in the reality scene based on the shooting pose data, the attribute information, and the pose data of the virtual object in the three-dimensional scene model configured to representing the reality scene.

8. The method of claim 7 , wherein the pre-stored second neural network model is trained according to the following step:

training the pre-stored second neural network model based on multiple sample images obtained by shooting of the reality scene in advance and attribute information corresponding to each of the multiple sample images.

9. The method of claim 1 , wherein after acquiring the shooting pose data of the AR device, the method further comprises:

acquiring a preset identifier of a reality scene shot by the AR device; and

determining additional virtual object information corresponding to the reality scene shot by the AR device based on the preset identifier and a pre-stored mapping relationship between preset identifiers and the additional virtual object information,

wherein acquiring the presentation special effect information of the virtual object corresponding to the shooting pose data in the reality scene based on the shooting pose data and the pose data of the virtual object in the three-dimensional scene model configured to represent the reality scene comprises:

acquiring the presentation special effect information of the virtual object corresponding to the shooting pose data in the reality scene based on the shooting pose data, the additional virtual object information, and the pose data of the virtual object in the three-dimensional scene model configured to represent the reality scene.

10. The method of claim 1 , wherein after displaying the AR scene image through the AR device based on the presentation special effect information, the method further comprises:

acquiring a triggering operation for the virtual object displayed in the AR device, and updating the presentation special effect information presented in the AR scene image.

11. The method of claim 10 , wherein the virtual object comprises a target musical instrument; and acquiring the triggering operation for the virtual object displayed in the AR device and updating the presentation special effect information presented in the AR scene image comprises:

acquiring the triggering operation for the virtual object displayed in the AR device, and controlling the AR device to update a sound playing effect of the virtual object to a sound playing effect corresponding to the triggering operation.

12. The method of claim 10 , wherein the virtual object comprises a target musical instrument, and in condition that there are multiple AR devices, acquiring the triggering operation for the virtual object displayed in the AR device and updating the presentation special effect information presented in the AR scene image comprises:

acquiring multiple triggering operations for a same virtual object displayed in the multiple AR devices, and controlling the multiple AR devices to update a sound playing effect of the same virtual object to a mixed sound playing effect corresponding to the multiple triggering operations for the same virtual object.

13. The method of claim 10 , wherein the virtual object comprises a target musical instrument, and in condition that there are multiple AR devices, acquiring the triggering operation for the virtual object displayed in the AR device and updating the presentation special effect information presented in the AR scene image comprises:

acquiring a triggering operation for at least one virtual object displayed in at least one AR device of the multiple AR devices, and controlling the at least one AR device to update a sound playing effect of the at least one virtual object to a mixed sound playing effect corresponding to the triggering operation for the at least one virtual object.

14. An electronic device, comprising a processor, a memory and a bus, wherein the memory is configured to store machine-readable instructions executable by the processor; and during operation of the electronic device, the processor communicates with the memory via the bus; and wherein the machine-readable instructions, when being executed by the processor, cause the processor to execute the operations of:

acquiring shooting pose data of an Augmented Reality (AR) device;

acquiring presentation special effect information of a virtual object corresponding to the shooting pose data in a reality scene based on the shooting pose data and pose data of the virtual object in a three-dimensional scene model configured to represent the reality scene; and

displaying an AR scene image through the AR device based on the presentation special effect information

wherein the three-dimensional scene model is generated in the following manner:

acquiring multiple reality scene images corresponding to the reality scene; and

generating the three-dimensional scene model based on the multiple reality scene images;

wherein the operation of generating the three-dimensional scene model based on the multiple reality scene images comprises:

extracting multiple feature points from each reality scene image of the multiple reality scene images; and

generating the three-dimensional scene model based on the multiple feature points and a pre-stored three-dimensional sample image matched with the reality scene, wherein the pre-stored three-dimensional sample image comprises a pre-stored three-dimensional image representing a morphology feature of the reality scene.

15. The electronic device of claim 14 , wherein the operation of acquiring the presentation special effect information of the virtual object corresponding to the shooting pose data in the reality scene based on the shooting pose data and the pose data of the virtual object in the three-dimensional scene model configured to represent the reality scene comprises:

acquiring the presentation special effect information of the virtual object corresponding to the shooting pose data based on the shooting pose data, the pose data of the virtual object in the three-dimensional scene model, and the three-dimensional scene model.

16. A non-transitory computer-readable storage medium, having stored therein a computer program, wherein the computer program, when being executed by a processor, causes the processor to implement operations of:

acquiring shooting pose data of an Augmented Reality (AR) device;

acquiring presentation special effect information of a virtual object corresponding to the shooting pose data in a reality scene based on the shooting pose data and pose data of the virtual object in a three-dimensional scene model configured to represent the reality scene; and

displaying an AR scene image through the AR device based on the presentation special effect information,

wherein the three-dimensional scene model is generated in the following manner:

acquiring multiple reality scene images corresponding to the reality scene; and

generating the three-dimensional scene model based on the multiple reality scene images;

wherein the operation of generating the three-dimensional scene model based on the multiple reality scene images comprises:

extracting multiple feature points from each reality scene image of the multiple reality scene images; and

generating the three-dimensional scene model based on the multiple feature points and a pre-stored three-dimensional sample image matched with the reality scene, wherein the pre-stored three-dimensional sample image comprises a pre-stored three-dimensional image representing a morphology feature of the reality scene.