IP Library › Granted Patent US 9,643,085

Granted Patent B2

US 9,643,085 · App. 13/277,561 · Granted May 9, 2017

Computer-readable storage medium, information processing system, and information processing method for controlling a virtual object using attitude data

Inventors: Hiroshi Matsunaga (Kyoto, JP); Yugo Hayashi (Kyoto, JP); Kazuya Sumaki (Kyoto, JP); Keiichi Minato (Kyoto, JP)

Assignee: Nintendo Co., Ltd.

A63F13/42 A63F13/214 A63F13/26 A63F13/5255 A63F13/5258 A63F13/211 A63F13/218 A63F2300/105 A63F2300/1068 A63F2300/301 A63F2300/403 A63F2300/6676 A63F2300/6684

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 9,643,085
App. No.: 13/277,561
Granted: May 9, 2017
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

An object arranged in a virtual world is caused to move in accordance with data based on a load applied to a load detection device. Then, based on attitude data outputted from a portable display device and a position of the object in the virtual world, a first virtual camera for generating an image of the virtual world is controlled, and a first image representing the virtual world viewed from the first virtual camera is displayed on the portable display device.

Claims (122)

1. A non-transitory computer-readable storage medium comprising an information processing program executed on a computer of an information processing apparatus which is capable of displaying an image on a separately housed portable display device which outputs at least attitude data based on an attitude of a body of the portable display device, wherein the information processing program, when executed, causes the computer to:

obtain data based on a load applied to a load detection device;

cause an object arranged in a virtual world to move based on the obtained data;

render, using processing circuitry of the information processing apparatus, a first image from a first virtual camera;

control the first virtual camera, which generates the first image of the virtual world, based on the attitude data, corresponding to the attitude of the body of the portable display device, and a position of the object in the virtual world;

render, using the processing circuitry of the information processing apparatus, a second image from a second virtual camera having a different viewpoint than the first virtual camera;

transmit, from the information processing apparatus to the separately housed portable display device, an encoded signal including the first image representing the virtual world viewed from the first virtual camera and rendered using the processing circuitry of the information processing apparatus; and

output, from the information processing apparatus to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera and rendered using the processing circuitry of the information processing apparatus.

2. The non-transitory computer-readable storage medium according to claim 1 , wherein

the information processing program further causes the computer to calculate an attitude of the portable display device, based on the attitude data, and

control the first virtual camera based on the calculated attitude of the portable display device and the position of the object.

3. The non-transitory computer-readable storage medium according to claim 2 , wherein

at least one of a position and an attitude of the first virtual camera in the virtual world is controlled based on the attitude of the portable display device.

4. The non-transitory computer-readable storage medium according to claim 3 , wherein

an attitude of the portable display device is calculated with respect to a predetermined direction in a real space, and

at least one of the position and the attitude of the first virtual camera is controlled, with reference to a direction which corresponds to the predetermined direction and is set in the virtual world, based on the attitude of the portable display device with respect to the predetermined direction in the real space.

5. The non-transitory computer-readable storage medium according to claim 4 , wherein

the predetermined direction is a direction of gravity in the real space, and an attitude of the portable display device is calculated with respect to the direction of gravity, and

at least one of the position and the attitude of the first virtual camera is controlled, with reference to a direction of gravity set in the virtual world, based on the attitude of the portable display device with respect to the direction of gravity in the real space.

6. The non-transitory computer-readable storage medium according to claim 5 , wherein

at least an attitude in which the portable display device rotates around the direction of gravity in the real space is calculated, and

the attitude of the first virtual camera is controlled by rotating the first virtual camera around the direction of gravity set in the virtual world, based on the attitude in which the portable display device rotates around the direction of gravity in the real space.

7. The non-transitory computer-readable storage medium according to claim 2 , wherein

at least one of a position and an attitude of the first virtual camera is controlled by controlling a line-of-sight direction of the first virtual camera in the virtual world, the line-of-sight direction being based on the attitude of the portable display device.

8. The non-transitory computer-readable storage medium according to claim 2 , wherein

an attitude of the first virtual camera is controlled so as to be the same as the attitude of the portable display device.

9. The non-transitory computer-readable storage medium according to claim 2 , wherein

the first virtual camera is controlled such that the object is included in the first image.

10. The non-transitory computer-readable storage medium according to claim 2 , wherein

the object is caused to move in a direction based on a line-of-sight direction of the first virtual camera.

11. The non-transitory computer-readable storage medium according to claim 2 , wherein

the portable display device includes at least one of a gyro sensor and an acceleration sensor, and

an attitude of the portable display device is calculated based on data outputted from at least one of the gyro sensor and the acceleration sensor.

12. The non-transitory computer-readable storage medium according to claim 2 , wherein

image data representing the first image is transmitted to the portable display device, and

the portable display device is configured to:

obtain image data outputted from the information processing apparatus; and

display the first image represented by the obtained image data.

13. The non-transitory computer-readable storage medium according to claim 12 , wherein

the information processing program further causes the computer to generate compressed image data by compressing the image data representing the first image,

output the generated compressed image data to the portable display device, and

obtain the compressed image data outputted from the information processing apparatus, wherein

the portable display device is further configured to decompress the compressed image data to obtain image data representing the first display image, and

display the first image represented by the obtained and decompressed image data.

14. The non-transitory computer-readable storage medium according to claim 1 , wherein

the information processing program further causes the computer to set, based on the position of the object in the virtual world, the second virtual camera for generating an image of the virtual world, in a position different from the position of the first virtual camera such that the object is included in the second image.

15. The non-transitory computer-readable storage medium according to claim 14 , wherein

the second virtual camera is set in a position apart from the object by a distance longer than the distance between the object and the first virtual camera, and

a range of the virtual world, which range is broader than the range of the virtual world as the first image, is displayed on the stationary display device as the second image.

16. The non-transitory computer-readable storage medium according to claim 14 , wherein

the second virtual camera is set in a position in the virtual world from which the second virtual camera views the object in a manner of bird's eye viewing, and

an image in which the object arranged in the virtual world is viewed in a manner of bird's eye viewing is displayed on the stationary display device as the second image.

17. The non-transitory computer-readable storage medium according to claim 1 , wherein

a line-of-sight direction of the second virtual camera in the virtual world is changed in accordance with a change in a line-of-sight direction of the first virtual camera in the virtual world.

18. The non-transitory computer-readable storage medium according to claim 1 , wherein the first and second image are rendered using a graphics processing unit of the information processing apparatus.

19. The non-transitory computer-readable storage medium according to claim 1 , wherein the signal transmitted from the information processing apparatus to the separately housed portable display device is encoded using a codec.

20. A non-transitory computer-readable storage medium comprising an information processing program executed on a computer of an information processing apparatus which is capable of displaying an image on a separately housed portable display device which outputs at least attitude data according to an attitude of a body of the portable display device, wherein the information processing program, when executed, causes the computer to:

obtain data based on a load applied to a load detection device;

render, using processing circuitry of the information processing apparatus, a first image from a first virtual camera;

control the first virtual camera which generates the first image of the virtual world, based on the obtained data and the attitude data, corresponding to the attitude of the body of the portable display device;

render, using the processing circuitry of the information processing apparatus, a second image from a second virtual camera having a different viewpoint than the first virtual camera;

21. An information processing system including a portable display device, a load detection device, and an information processing apparatus capable of displaying an image on the portable display device, wherein

the load detection device configured to output, to the information processing apparatus, data based on a load applied to the load detection device,

the portable display device configured to output, to the separately housed information processing apparatus, at least attitude data according to an attitude of the portable display device,

the information processing apparatus configured to:

obtain data outputted from the load detection device;

cause an object arranged in the virtual world to move, based on the obtained data;

render a first image from a first virtual camera using processing circuitry of the information processing apparatus;

control the first virtual camera which generates the first image of the virtual world, based on the attitude data, corresponding to the attitude of the portable display device, and a position of the object in the virtual world;

render a second image from a second virtual camera having a different viewpoint than the first virtual camera using the processing circuitry of the information processing apparatus;

transmit, to the separately housed portable display device, an encoded signal including the first image representing the virtual world viewed from the first virtual camera and rendered using the processing circuitry of the information processing apparatus; and

output, to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera and rendered using the processing circuitry of the information processing apparatus, and

the portable display device further configured to:

obtain the data of the first image transmitted from the information processing apparatus; and

display the first image represented by the obtained image data.

22. An information processing system including a portable display device, a load detection device, and an information processing apparatus capable of displaying an image on the portable display device, wherein

the load detection device configured to output, to the information processing apparatus, data based on a load applied to the load detection device,

the portable display device configured to output, to the separately housed information processing apparatus, at least attitude data according to an attitude of the portable display device,

the information processing apparatus configured to:

obtain data outputted from the load detection device;

render a first image from a first virtual camera using processing circuitry of the information processing apparatus;

control the first virtual camera which generates the first image of the virtual world, based on the obtained data, and the attitude data, corresponding to the attitude of the portable display device;

render a second image from a second virtual camera having a different viewpoint than the first virtual camera using the processing circuitry of the information processing apparatus;

the portable display device further configured to:

obtain the data of the first image transmitted from the information processing apparatus; and

display the first image represented by the obtained image data.

23. An information processing method executed by one processor included in an information processing system that includes at least one information processing apparatus capable of displaying an image on a separately housed portable display device that outputs at least attitude data according to an attitude of a body of the portable display device, or by a plurality of processors included in the information processing system in a cooperative manner, the information processing method comprising:

obtaining data based on a load applied to a load detection device;

causing an object arranged in a virtual world to move based on the obtained data;

rendering, using processing circuitry of the information processing apparatus, a first image from a first virtual camera;

controlling the first virtual camera which generates the first image of the virtual world, based on the attitude data, corresponding to the attitude of the body of the portable display device, and a position of the object in the virtual world;

rendering, using the processing circuitry of the information processing apparatus, a second image from a second virtual camera having a different viewpoint than the first virtual camera;

transmitting, from the information processing apparatus to the separately housed portable display device, an encoded signal including the first image representing the virtual world viewed from the first virtual camera and rendered using the processing circuitry of the information processing apparatus; and

outputting, from the information processing apparatus to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera and rendered using the processing circuitry of the information processing apparatus.

24. An information processing method executed by one processor included in an information processing system that includes at least one information processing apparatus capable of displaying an image on a separately housed portable display device that outputs at least attitude data according to an attitude of a body of the portable display device, or by a plurality of processors included in the information processing system in a cooperative manner, the information processing method comprising:

obtaining data based on a load applied to a load detection device;

rendering, using processing circuitry of the information processing apparatus, a first image from a first virtual camera;

controlling the first virtual camera which generates the first image of the virtual world, based on the obtained data and the attitude data, corresponding to the attitude of the body of the portable display device;

rendering, using the processing circuitry of the information processing apparatus, a second image from a second virtual camera having a different viewpoint than the first virtual camera;

25. An information processing apparatus, comprising:

a processing system having at least one processor, the processing system configured to:

obtain data based on a load applied to a load detection device;

cause an object arranged in a virtual world to move based on the obtained data;

render a first image of a virtual world from a first virtual camera using the processing system of the information processing apparatus;

control the first virtual camera, which generates the first image of the virtual world, based on attitude data, corresponding to an attitude of a body of a separately housed portable display device, and a position of the object in the virtual world;

render a second image from a second virtual camera having a different viewpoint than the first virtual camera using the processing system of the information processing apparatus;

transmit, from the information processing apparatus to the separately housed portable display device, a signal including the first image representing the virtual world viewed from the first virtual camera; and

26. An information processing apparatus, comprising:

a processing system having at least one processor, the processing system configured to:

obtain data based on a load applied to a load detection device;

render a first image of a virtual world from a first virtual camera using the processing system of the information processing apparatus;

control the first virtual camera which generates the first image of the virtual world, based on the obtained data and attitude data, corresponding to an attitude of a body of a separately housed portable display device;

render a second image from a second virtual camera having a different viewpoint than the first virtual camera using the processing system of the information processing apparatus;

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2011

From: MATSUNAGA, HIROSHI; HAYASHI, YUGO; SUMAKI, KAZUYA; MINATO, KEIICHI

To: NINTENDO CO., LTD.

Reel/Frame 027093/0064 →

Priority Claims (2)

JP 2011-050039 · Mar 8, 2011 · national

JP 2011-083453 · Apr 5, 2011 · national

Continuity (1)

Related Publication 20120229516A1 · Sep 13, 2012