Image drawing method performed by remote-side apparatus and local-side apparatus
An image drawing method performed by first and second computers is provided. The image drawing method includes: transmitting, by the second computer, a streaming video that includes a pen stroke image generated by a first rendering process; generating, by the first computer, a pen stroke image by performing a second rendering process, which is shared with the first rendering process, in parallel with the first rendering process; receiving, by the first computer, the streaming video from the second computer via communication that incurs a delay; compositing, by the first computer, the pen stroke image into the streaming video; and outputting, by the first computer, the streaming video. The image drawing method enables alignment of appearances between a provisional image and the pen stroke image that appears within the streaming video.
1 . An image processing method performed by a remote-side apparatus and a local-side apparatus, the method comprising:
transmitting, by the remote-side apparatus, a streaming video that includes an image generated by a first rendering process;
generating, by the local-side apparatus, a provisional image by performing a second rendering process shared with the first rendering process in parallel with the first rendering process;
receiving, by the local-side apparatus, the streaming video from the remote-side apparatus via communication that incurs a delay;
compositing, by the local-side apparatus, the provisional image into the streaming video;
outputting, by the local-side apparatus, the streaming video;
transmitting, by the remote-side apparatus, to the local-side apparatus, information indicating, from among positions indicated by a pen and sequentially received from the local-side apparatus, a position indicated by the pen for which the first rendering process has started;
erasing, by the local-side apparatus, from the provisional image, a portion corresponding to the position indicated by the pen for which start of the first rendering process is indicated by the information from the remote-side apparatus;
displaying, by the local-side apparatus, on a display, a setting screen that sets a generation method for a second pen stroke image,
wherein the second rendering process uses the generation method set using the setting screen to draw the second pen stroke image until sharing between the first rendering process and the second rendering process completes,
wherein the displaying displays the setting screen in response to a state of a predetermined switch of the pen having changed to a first state, and
wherein the local-side apparatus does not notify the remote-side apparatus that the state of the predetermined switch of the pen has changed to the first state.
2 . The image processing method according to claim 1 , wherein the delay includes at least one of a network delay, an image compression encoding delay, or a buffering delay.
3 . The image processing method according to claim 1 , comprising:
detecting, by the local-side apparatus, a plurality of positions indicated by-a the pen; and
transmitting, by the local-side apparatus, the positions indicated by the pen to the remote-side apparatus, and
wherein each of the first rendering process and the second rendering process draws a pen stroke image based on the positions indicated by the pen.
4 . The image processing method according to claim 3 , wherein sharing between the first rendering process and the second rendering process includes sharing, between the local-side apparatus and the remote-side apparatus, a pen tip attribute referred to when drawing the pen stroke image.
5 . The image processing method according to claim 4 , wherein the pen tip attribute includes a shape of a pen tip of the pen.
6 . The image processing method according to claim 4 , wherein the pen tip attribute includes a drawing color of the pen stroke image.
7 . The image processing method according to claim 4 , wherein the pen tip attribute includes a function used to derive a pen stroke attribute based on information pertaining to the pen.
8 . The image processing method according to claim 7 , wherein:
the information pertaining to the pen includes a pen pressure value, and
the pen stroke attribute includes a line width.
9 . The image processing method according to claim 4 , wherein the pen tip attribute includes a random number seed used with particle scattering to draw the pen stroke image.
10 . The image processing method according to claim 4 , wherein the local-side apparatus, after detecting the pen, at least periodically receives information pertaining to the pen tip attribute from the remote-side apparatus.
11 . The image processing method according to claim 3 , wherein sharing between the first rendering process and the second rendering process includes sharing, between the local-side apparatus and the remote-side apparatus, a drawing region for a pen stroke.
12 . The image processing method according to claim 11 , wherein the second rendering process draws the pen stroke image within the drawing region.
13 . The image processing method according to claim 11 , wherein the local-side apparatus performs the second rendering process in a case where a pen-down position for the pen is within the drawing region.
14 . The image processing method according to claim 11 , wherein the local-side apparatus, after making the drawing region be fixed, performs the second rendering process in a duration from a pen-down to a pen-up, for the pen.
15 . The image processing method according to claim 3 , wherein a second pen stroke image drawn by the second rendering process has a shape matching a first pen stroke image drawn by the first rendering process, and the second pen stroke image differs from the first pen stroke image in at least one of a pattern or a color.
16 . The image processing method according to claim 15 , comprising:
erasing, by the local-side apparatus, the provisional image composited with the streaming video,
wherein, from a pen stroke image displayed at a time of the erasing, a portion corresponding to the first pen stroke image differs from a portion corresponding to the second pen stroke image in at least one of a pattern or a color.
17 . The image processing method according to claim 16 , wherein the color includes a degree of transparency.
18 . The image processing method according to claim 15 , wherein the local-side apparatus uses a pattern resulting from mixing two or more colors to generate the second pen stroke image.
19 . The image processing method according to claim 15 , wherein the local-side apparatus determines a drawing color for the second pen stroke image based on information that specifies a drawing color for the first pen stroke image and is obtained from the remote-side apparatus.
20 . The image processing method according to claim 15 , wherein the local-side apparatus determines a drawing color for the second pen stroke image based on a background color for a drawing region.
21 . The image processing method according to claim 15 , wherein the local-side apparatus uses an outline to draw the second pen stroke image in a case where the first pen stroke image is not a stroke for drawing a line.
22 . The image processing method according to claim 3 , wherein a second pen stroke image drawn by the second rendering process has a trajectory matching a first pen stroke image drawn by the first rendering process, and the second pen stroke image differs from the first pen stroke image in shape.
23 . The image processing method according to claim 3 , comprising:
transmitting, by the local-side apparatus, a plurality of pieces of data pertaining to the pen including a position indicated by the pen to the remote-side apparatus;
storing, by the local-side apparatus, flag information indicating whether to transmit each of the plurality of pieces of data; and
controlling, the local-side apparatus, the flag information in accordance with information displayed by a display of the local-side apparatus,
wherein the local-side apparatus does not transmit, to the remote-side apparatus, data from among the plurality of pieces of data for which no transmission is indicated by the flag information.
24 . The image processing method according to claim 1 , comprising:
erasing, by the local-side apparatus, a portion of the provisional image for which a predetermined amount of time has elapsed after generation.
25 . The image processing method according to claim 24 , wherein the predetermined amount of time is set by the remote-side apparatus.
26 . The image processing method according to claim 24 , wherein the predetermined amount of time is set based on a length of time for the delay.
27 . The image processing method according to claim 1 , comprising:
erasing, by the local-side apparatus, a portion of the provisional image that has appeared within the streaming video.
28 . The image processing method according to claim 1 , wherein the remote-side apparatus and the local-side apparatus are computers that are separate from one another.
29 . The image processing method according to claim 1 , wherein:
the local-side apparatus is a central processing unit in a computer, and
the remote-side apparatus is a virtual machine that operates within the local-side apparatus.
30 . The image processing method according to claim 1 , wherein:
the local-side apparatus is an integrated circuit provided within a computer, and
the remote-side apparatus is a central processing unit in the computer.
31 . The image processing method according to claim 1 , wherein:
the local-side apparatus is a central processing unit in a computer, and
the remote-side apparatus is an application that operates on a web browser.
32 . The image processing method according to claim 1 , wherein:
the local-side apparatus includes a pen input apparatus, and
the generating the provisional image is performed by an integrated circuit within the pen input apparatus.
33 . The image processing method according to claim 32 , wherein:
the pen input apparatus includes a display apparatus, and
the method further comprises:
outputting, by the pen input apparatus, separately from the compositing the provisional image, the provisional image to the display apparatus.
34 . The image processing method according to claim 32 , wherein:
the local-side apparatus includes a computer, and
the method further comprises:
receiving, by the computer, information indicating a pen tip attribute from the remote-side apparatus; and
switching, by the computer, a location for generating the provisional image from the pen input apparatus to the computer in a case where the pen input apparatus does not support the pen tip attribute received from the remote-side apparatus.