IP Library Granted Patent US 12,536,586
Granted Patent B2
US 12,536,586 · App. 17/493,455 · Granted Jan 27, 2026

System and method for an agriculture exchange for agriculture units

Inventor: Erik Mowery Simpson (Houston, TX)
Assignee: CIRCLESX LLC
G06Q40/04G06F16/212G06F16/248G06Q50/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,536,586
App. No.
17/493,455
Filed
Oct 4, 2021
Granted
Jan 27, 2026
Kind
B2
Art Unit
3628
USPC
705/37
Abstract

Implementations of various methods and systems to transform agriculture units with two waypoints or a destination waypoint or a series sequence of waypoints into objects which have associated price—time priority queues for transformed agriculture units. The present disclosed invention relates to combining the concepts of objected oriented programming, market price queues, agriculture, navigation systems and social networking and transportation as a fungible asset class or agriculture linked to transportation as an open market.

Claims (65)

1 . A method, comprising:

receiving, by a networked computing system, origin location data and destination location data from a plurality of users, wherein the origin location data corresponds to geographic origins and the destination location data corresponds to geographic destinations;

determining, by one or more processors of the networked computing system, a plurality of virtual hubs based on the origin location data and the destination location data, wherein the plurality of virtual hubs comprises one or more origin virtual hubs determined based on the origin location data and one or more destination virtual hubs determined based on the destination location data, and wherein the plurality of virtual hubs are dynamically updated in response to changes to at least one of the origin location data and the destination location data;

storing, in one or more database servers of the networked computing system, virtual hub location data corresponding to the one or more origin virtual hubs and the one or more destination virtual hubs;

determining, by the one or more processors of the networked computing system, a plurality of virtual hub routes based on the plurality of virtual hubs, wherein a respective virtual hub route corresponds to a geographic route from a location corresponding to a respective origin virtual hub to a location corresponding to a respective destination virtual hub;

receiving, by the networked computing system, agricultural attribute data from the plurality of users, wherein the agricultural attribute data corresponds to one or more attributes of one or more agricultural products available for purchase from the plurality of users using the plurality of virtual hub routes;

generating, by the one or more processors of the networked computing system, an agricultural market platform for one or more agricultural units based on the plurality of virtual hub routes and the agricultural attribute data, wherein:

the one or more agricultural units correspond to the one or more agricultural products available for purchase from the plurality of users using the plurality of virtual hub routes;

the agricultural market platform corresponds to market depth data, the market depth data comprising data indicating at least a bid price or offer price provided by a respective user for a respective agricultural unit; and

the agricultural market platform provides real-time transaction capabilities between the plurality of users for the purchase and sale of the one or more agricultural products available for purchase; and

storing, in the one or more database servers of the networked computing system, the agricultural market platform.

2 . The method of claim 1 , further comprising:

generating the one or more agricultural units for at least a first subset of the plurality of users based on the one or more agricultural products available for purchase from the plurality of users using the plurality of virtual hub routes, wherein:

the agricultural attribute data comprises data corresponding to product type, product quantity, product specification, product price, product availability, or combinations thereof.

3 . The method of claim 1 , further comprising:

detecting, at respective user interfaces of the plurality of users, respective user network of the respective user interfaces using facial recognition or fingerprint recognition for authentication, wherein:

the one or more agricultural products comprise one or more products corresponding to agricultural livestock, one or more products corresponding to agricultural produce, or combinations thereof.

4 . The method of claim 3 , further comprising:

generating one or more graphical layers on respective user interfaces of at least one of first and second users of the plurality of users, wherein:

the one or more graphical layers are configured to be superimposed on a respective navigational map interface of the respective user interfaces; and

the one or more graphical layers comprise at least one or more of: buttons, icons, settings tables, or menus configured for selection or manipulation by at least one of the first and second users;

the one or more products corresponding to the agricultural livestock comprise one or more products corresponding to chicken, beef, lamb, goat, fish, horse, pork, or combinations thereof; and

the one or more products corresponding to the agricultural produce comprise one or more products corresponding to apples, oranges, lettuce, spinach, grapes, potatoes, corn, wheat, or combinations thereof.

5 . The method of claim 1 , further comprising receiving transaction input data from the plurality of users, wherein the transaction input data comprises data corresponding to acceptance of at least the bid price or the offer price.

6 . The method of claim 5 , further comprising generating an interface on a computing device, wherein the interface is configured to receive the transaction input data, and wherein the transaction input data comprises fixed input data, visual input data, audio input data, or combinations thereof.

7 . The method of claim 1 , further comprising generating a priority queue of the one or more agricultural units using the agricultural market platform, wherein the priority queue orders the one or more agricultural units based on product type, product quantity, product specification, product price, product availability, or combinations thereof.

8 . The method of claim 1 , wherein the respective virtual hub route comprises one or more additional locations along the geographic route, and wherein the one or more additional locations correspond to one or more additional virtual hubs.

9 . A computing system, comprising:

one or more processors; and

one or more memory comprising program instructions executable by the one or more processors to:

receive origin location data and destination location data from a plurality of users, wherein the origin location data corresponds to geographic origins and the destination location data corresponds to geographic destinations;

determine a plurality of virtual hubs based on the origin location data and the destination location data, wherein the plurality of virtual hubs comprises one or more origin virtual hubs determined based on the origin location data and one or more destination virtual hubs determined based on the destination location data, and wherein the plurality of virtual hubs are dynamically updated in response to changes to at least one of the origin location data and the destination location data;

store, in one or more database servers of the computing system, virtual hub location data corresponding to the one or more origin virtual hubs and the one or more destination virtual hubs;

determine a plurality of virtual hub routes based on the plurality of virtual hubs, wherein a respective virtual hub route corresponds to a geographic route from a location corresponding to a respective origin virtual hub to a location corresponding to a respective destination virtual hub;

receive agricultural attribute data from the plurality of users, wherein the agricultural attribute data corresponds to one or more attributes of one or more agricultural products available for purchase from the plurality of users using the plurality of virtual hub routes;

generate an agricultural market platform for one or more agricultural units based on the plurality of virtual hub routes and the agricultural attribute data, wherein:

the one or more agricultural units correspond to the one or more agricultural products available for purchase from the plurality of users using the plurality of virtual hub routes;

the agricultural market platform corresponds to market depth data, the market depth data comprising data indicating at least a bid price or offer price provided by a respective user for a respective agricultural unit; and

the agricultural market platform provides real-time transaction capabilities between the plurality of users for the purchase and sale of the one or more agricultural products available for purchase; and

store, in the one or more database servers of the computing system, the agricultural market platform.

10 . The computing system of claim 9 , wherein the agricultural attribute data comprises data corresponding to product type, product quantity, product specification, product price, product availability, or combinations thereof.

11 . The computing system of claim 9 , wherein the one or more agricultural products comprise one or more products corresponding to agricultural livestock, one or more products corresponding to agricultural produce, or combinations thereof.

12 . The computing system of claim 11 , wherein:

the one or more products corresponding to the agricultural livestock comprise one or more products corresponding to chicken, beef, lamb, goat, fish, horse, pork, or combinations thereof; and

the one or more products corresponding to the agricultural produce comprise one or more products corresponding to apples, oranges, lettuce, spinach, grapes, potatoes, corn, wheat, or combinations thereof.

13 . The computing system of claim 9 , wherein the one or more memory further comprises program instructions executable by the one or more processors to receive transaction input data from the plurality of users, wherein the transaction input data comprises data corresponding to acceptance of at least the bid price or the offer price.

14 . The computing system of claim 13 , wherein the one or more memory further comprises program instructions executable by the one or more processors to generate an interface on a computing device, wherein the interface is configured to receive the transaction input data, and wherein the transaction input data comprises fixed input data, visual input data, audio input data, or combinations thereof.

15 . The computing system of claim 9 , wherein the one or more memory further comprises program instructions executable by the one or more processors to generate a priority queue of the one or more agricultural units using the agricultural market platform, wherein the priority queue orders the one or more agricultural units based on product type, product quantity, product specification, product price, product availability, or combinations thereof.

16 . The computing system of claim 9 , wherein the respective virtual hub route comprises one or more additional locations along the geographic route, and wherein the one or more additional locations correspond to one or more additional virtual hubs.

17 . A non-transitory computer-readable medium having stored thereon a plurality of computer-executable instructions which, when executed by a computer, cause the computer to:

receive origin location data and destination location data from a plurality of users, wherein the origin location data corresponds to geographic origins and the destination location data corresponds to geographic destinations;

determine a plurality of virtual hubs based on the origin location data and the destination location data, wherein the plurality of virtual hubs comprises one or more origin virtual hubs determined based on the origin location data and one or more destination virtual hubs determined based on the destination location data, and wherein the plurality of virtual hubs are dynamically updated in response to changes to at least one of the origin location data and the destination location data;

store, in one or more database servers of a computing system, virtual hub location data corresponding to the one or more origin virtual hubs and the one or more destination virtual hubs;

determine a plurality of virtual hub routes based on the plurality of virtual hubs, wherein a respective virtual hub route corresponds to a geographic route from a location corresponding to a respective origin virtual hub to a location corresponding to a respective destination virtual hub;

receive agricultural attribute data from the plurality of users, wherein the agricultural attribute data corresponds to one or more attributes of one or more agricultural products available for purchase from the plurality of users using the plurality of virtual hub routes;

generate an agricultural market platform for one or more agricultural units based on the plurality of virtual hub routes and the agricultural attribute data, wherein:

the one or more agricultural units correspond to the one or more agricultural products available for purchase from the plurality of users using the plurality of virtual hub routes;

the agricultural market platform corresponds to market depth data, the market depth data comprising data indicating at least a bid price or offer price provided by a respective user for a respective agricultural unit; and

the agricultural market platform provides real-time transaction capabilities between the plurality of users for the purchase and sale of the one or more agricultural products available for purchase; and

store, in the one or more database servers of the computing system, the agricultural market platform.

18 . The non-transitory computer-readable medium of claim 17 , wherein the agricultural attribute data comprises data corresponding to product type, product quantity, product specification, product price, product availability, or combinations thereof.

19 . The non-transitory computer-readable medium of claim 17 , wherein the one or more agricultural products comprise one or more products corresponding to agricultural livestock, one or more products corresponding to agricultural produce, or combinations thereof.

20 . The non-transitory computer-readable medium of claim 19 , wherein:

the one or more products corresponding to the agricultural livestock comprise one or more products corresponding to chicken, beef, lamb, goat, fish, horse, pork, or combinations thereof; and

the one or more products corresponding to the agricultural produce comprise one or more products corresponding to apples, oranges, lettuce, spinach, grapes, potatoes, corn, wheat, or combinations thereof.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 25, 2021
From: SIMPSX TECHNOLOGIES LLC
To: CIRCLESX LLC
Reel/Frame 058251/0180 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 12, 2021
From: SIMPSON, ERIK
To: SIMPSON HOLDINGS, INC.
Reel/Frame 058101/0172 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 12, 2021
From: SIMPSON HOLDINGS, INC.
To: SIMPSX TECHNOLOGIES LLC
Reel/Frame 058101/0175 →
Continuity (13)
Continuation 16290278 · Mar 1, 2019
Continuation 16274490 · Feb 13, 2019
Continuation In Part 16258658 · Jan 27, 2019
Continuation In Part 16257032 · Jan 24, 2019
Continuation In Part 16242967 · Jan 8, 2019
Continuation In Part 16242981 · Jan 8, 2019
Continuation In Part 16239485 · Jan 3, 2019
Continuation In Part 16183647 · Nov 7, 2018
Continuation In Part 16167525 · Oct 22, 2018
Continuation In Part 15877393 · Jan 23, 2018
Continuation In Part 15484059 · Apr 10, 2017
Continuation In Part 15266326 · Sep 15, 2016
Related Publication 20220027997A1 · Jan 27, 2022
References Cited (378)
US 1008279A · Leuthesser · 1911 [cited by applicant]
US D209710S · Bruce · 1967 [cited by applicant]
US D318073S · Jang · 1991 [cited by applicant]
US 5249259A · Harvey · 1993 [cited by applicant]
US 5751245A · Janky et al. · 1998 [cited by applicant]
US 5948040A · DeLorme · 1999 [cited by applicant]
US 6240396B1 · Walker et al. · 2001 [cited by applicant]
US D453945S · Shan · 2002 [cited by applicant]
US 6400996B1 · Hoffberg · 2002 [cited by applicant]
US D460952S · Kataoka · 2002 [cited by applicant]
US D468738S · Lin · 2003 [cited by applicant]
US D469089S · Lin · 2003 [cited by applicant]
US 7010472B1 · Vasey-Glandon et al. · 2006 [cited by applicant]
US D590396S · Lo · 2009 [cited by applicant]
US 7584123B1 · Karonis et al. · 2009 [cited by applicant]
US D628171S · Hakopian · 2010 [cited by applicant]
US D638879S · Suto · 2011 [cited by applicant]
US 8065191B2 · Senior · 2011 [cited by applicant]
US D650385S · Chiu · 2011 [cited by applicant]
US 9213957B2 · Stefik et al. · 2015 [cited by applicant]
US 9274540B2 · Anglin et al. · 2016 [cited by applicant]
US 9292764B2 · Yun et al. · 2016 [cited by applicant]
US 9436923B1 · Sriram et al. · 2016 [cited by applicant]
US 9450817B1 · Bahadur et al. · 2016 [cited by applicant]
US D772828S · Kusumoto · 2016 [cited by applicant]
US 9665983B2 · Spivack · 2017 [cited by applicant]
US 9952042B2 · Abovitz et al. · 2018 [cited by applicant]
US D832355S · Castro · 2018 [cited by applicant]
US 10216367B1 · Patel · 2019 [cited by applicant]
US 10395332B1 · Konrardy et al. · 2019 [cited by applicant]
US 10403050B1 · Beall et al. · 2019 [cited by applicant]
US 10408489B1 · Trishaun et al. · 2019 [cited by applicant]
US 10452978B2 · Shazeer et al. · 2019 [cited by applicant]
US 10533850B2 · Abovitz et al. · 2020 [cited by applicant]
US 10737585B2 · Chaudhary et al. · 2020 [cited by applicant]
US D896315S · Castro · 2020 [cited by applicant]
US 10832337B1 · Floyd et al. · 2020 [cited by applicant]
US D903657S · Catania · 2020 [cited by applicant]
US D903658S · Catania · 2020 [cited by applicant]
US D903659S · Catania · 2020 [cited by applicant]
US 10872381B1 · Leise et al. · 2020 [cited by applicant]
US D910758S · Leong · 2021 [cited by applicant]
US 11138827B2 · Simpson · 2021 [cited by applicant]
US 11183080B2 · Wolf et al. · 2021 [cited by applicant]
US D938375S · Zhang · 2021 [cited by applicant]
US 11215466B2 · Simpson · 2022 [cited by applicant]
US 11288563B2 · Lee et al. · 2022 [cited by applicant]
US 11296897B2 · Endress et al. · 2022 [cited by applicant]
US 11298017B2 · Tran · 2022 [cited by applicant]
US 11298591B2 · Evancha · 2022 [cited by applicant]
US 11537953B2 · Beaurepaire · 2022 [cited by applicant]
US 11555709B2 · Simpson · 2023 [cited by applicant]
US 11586993B2 · Handler et al. · 2023 [cited by applicant]
US D980210S · Wu · 2023 [cited by applicant]
US 11651464B2 · Park · 2023 [cited by applicant]
US D993316S · Lin · 2023 [cited by applicant]
US 11704219B1 · Lerner et al. · 2023 [cited by applicant]
US 11722500B2 · Singh · 2023 [cited by applicant]
US 11734618B2 · Ogden · 2023 [cited by applicant]
US D1000137S · Shuster · 2023 [cited by applicant]
US D1007451S · Im et al. · 2023 [cited by applicant]
US D1024065S · Kim et al. · 2024 [cited by applicant]
US 20020004788A1 · Gros et al. · 2002 [cited by applicant]
US 20020017997A1 · Wall · 2002 [cited by applicant]
US 20020065738A1 · Riggs et al. · 2002 [cited by applicant]
US 20020065766A1 · Brown · 2002 [cited by examiner]
US 20020128952A1 · Melkomaian · 2002 [cited by applicant]
US 20030191725A1 · Ratliff · 2003 [cited by examiner]
US 20040115596A1 · Snyder et al. · 2004 [cited by applicant]
US 20040249742A1 · Laurent et al. · 2004 [cited by applicant]
US 20040254819A1 · Halim · 2004 [cited by applicant]
US 20050132070A1 · Redlich et al. · 2005 [cited by applicant]
US 20060104224A1 · Singh · 2006 [cited by applicant]
US 20060184321A1 · Kawakami · 2006 [cited by applicant]
US 20070005224A1 · Sutardja · 2007 [cited by applicant]
US 20070260723A1 · Cohen · 2007 [cited by applicant]
US 20080129490A1 · Linville et al. · 2008 [cited by applicant]
US 20080140557A1 · Bowlby et al. · 2008 [cited by applicant]
US 20080195428A1 · O'Sullivan · 2008 [cited by applicant]
US 20080195432A1 · Fell et al. · 2008 [cited by applicant]
US 20080262892A1 · Prager et al. · 2008 [cited by applicant]
US 20090221338A1 · Stewart · 2009 [cited by applicant]
US 20090309729A1 · Nichols · 2009 [cited by applicant]
US 20100042421A1 · Bai et al. · 2010 [cited by applicant]
US 20100081548A1 · Labedz · 2010 [cited by applicant]
US 20100208029A1 · Marti · 2010 [cited by applicant]
US 20100280748A1 · Mundinger et al. · 2010 [cited by applicant]
US 20100306078A1 · Hwang · 2010 [cited by applicant]
US 20100318373A1 · Harris · 2010 [cited by applicant]
US 20110025267A1 · Kamen et al. · 2011 [cited by applicant]
US 20110059693A1 · O'Sullivan · 2011 [cited by applicant]
US 20110098056A1 · Rhoads et al. · 2011 [cited by applicant]
US 20110106660A1 · Ajjarapu et al. · 2011 [cited by applicant]
US 20110184784A1 · Rudow · 2011 [cited by applicant]
US 20110191248A1 · Bishop · 2011 [cited by applicant]
US 20110202418A1 · Kempton et al. · 2011 [cited by applicant]
US 20120023032A1 · Visdomini · 2012 [cited by applicant]
US 20120072925A1 · Jenkins et al. · 2012 [cited by applicant]
US 20120075067A1 · Attaluri · 2012 [cited by applicant]
US 20120130556A1 · Marhoefer · 2012 [cited by applicant]
US 20120136527A1 · McQuade · 2012 [cited by applicant]
US 20120158762A1 · IwuchukWu · 2012 [cited by applicant]
US 20130035973A1 · Desai et al. · 2013 [cited by applicant]
US 20130132261A1 · Ebersole · 2013 [cited by applicant]
US 20130173326A1 · Anglin et al. · 2013 [cited by applicant]
US 20130268325A1 · Dembo · 2013 [cited by applicant]
US 20130275156A1 · Kinkaid et al. · 2013 [cited by applicant]
US 20140032034A1 · Raptopoulos · 2014 [cited by applicant]
US 20140038781A1 · Foley · 2014 [cited by applicant]
US 20140052500A1 · Vallapuzha et al. · 2014 [cited by applicant]
US 20140075528A1 · Matsuoka · 2014 [cited by applicant]
US 20140129302A1 · Amin et al. · 2014 [cited by applicant]
US 20140149157A1 · Shaam et al. · 2014 [cited by applicant]
US 20140162598A1 · Villa-Real · 2014 [cited by applicant]
US 20140180732A1 · Rotchin · 2014 [cited by applicant]
US 20140229258A1 · Seriani · 2014 [cited by applicant]
US 20140282586A1 · Shear et al. · 2014 [cited by applicant]
US 20140310149A1 · Singh · 2014 [cited by applicant]
US 20150006428A1 · Miller et al. · 2015 [cited by applicant]
US 20150058051A1 · Movshovich · 2015 [cited by applicant]
US 20150097864A1 · Alaniz · 2015 [cited by applicant]
US 20150154516A1 · Joachim · 2015 [cited by applicant]
US 20150161564A1 · Sweeney et al. · 2015 [cited by applicant]
US 20150178642A1 · Abboud · 2015 [cited by applicant]
US 20150198459A1 · MacNeille et al. · 2015 [cited by applicant]
US 20150206443A1 · Aylesworth et al. · 2015 [cited by applicant]
US 20150260474A1 · Rublowsky et al. · 2015 [cited by applicant]
US 20150348282A1 · Gibbon et al. · 2015 [cited by applicant]
US 20160018969A1 · Sundarraman et al. · 2016 [cited by applicant]
US 20160026253A1 · Bradski et al. · 2016 [cited by applicant]
US 20160034305A1 · Shear et al. · 2016 [cited by applicant]
US 20160041628A1 · Verma · 2016 [cited by applicant]
US 20160063436A1 · Coles · 2016 [cited by applicant]
US 20160117657A1 · Forbes, Jr. et al. · 2016 [cited by applicant]
US 20160148289A1 · Altschuler · 2016 [cited by applicant]
US 20160221935A1 · Jaworska-Maslanka · 2016 [cited by examiner]
US 20160224935A1 · Burnett · 2016 [cited by examiner]
US 20160253622A1 · Sriram · 2016 [cited by applicant]
US 20160253662A1 · Sriram · 2016 [cited by applicant]
US 20160298977A1 · Newlin · 2016 [cited by applicant]
US 20160307276A1 · Young · 2016 [cited by applicant]
US 20160307288A1 · Yehuda et al. · 2016 [cited by applicant]
US 20160364679A1 · Cao · 2016 [cited by applicant]
US 20170039770A1 · Lanier et al. · 2017 [cited by applicant]
US 20170046664A1 · Haldenby et al. · 2017 [cited by applicant]
US 20170046799A1 · Chan et al. · 2017 [cited by applicant]
US 20170046806A1 · Haldenby et al. · 2017 [cited by applicant]
US 20170048216A1 · Chow et al. · 2017 [cited by applicant]
US 20170053461A1 · Pal et al. · 2017 [cited by applicant]
US 20170146360A1 · Averbuch · 2017 [cited by applicant]
US 20170232300A1 · Tran et al. · 2017 [cited by applicant]
US 20170293881A1 · Narkulla · 2017 [cited by applicant]
US 20170293950A1 · Rathod · 2017 [cited by applicant]
US 20170318325A1 · Ortiz · 2017 [cited by applicant]
US 20170330274A1 · Conant, II et al. · 2017 [cited by applicant]
US 20170356749A1 · Shelby · 2017 [cited by applicant]
US 20170357914A1 · Tulabandhula · 2017 [cited by applicant]
US 20170373509A1 · Betzin · 2017 [cited by applicant]
US 20180012149A1 · Yust · 2018 [cited by applicant]
US 20180025417A1 · Brathwaite et al. · 2018 [cited by applicant]
US 20180046431A1 · Thagadur Shivappa et al. · 2018 [cited by applicant]
US 20180053226A1 · Hutton et al. · 2018 [cited by applicant]
US 20180068355A1 · Garry · 2018 [cited by applicant]
US 20180088455A1 · Cippant · 2018 [cited by applicant]
US 20180111494A1 · Penilla et al. · 2018 [cited by applicant]
US 20180117447A1 · Bao et al. · 2018 [cited by applicant]
US 20180121958A1 · Aist et al. · 2018 [cited by applicant]
US 20180129276A1 · Nguyen et al. · 2018 [cited by applicant]
US 20180140903A1 · Poure · 2018 [cited by applicant]
US 20180157999A1 · Arora · 2018 [cited by examiner]
US 20180165364A1 · Mehta et al. · 2018 [cited by applicant]
US 20180173742A1 · Liu et al. · 2018 [cited by applicant]
US 20180173800A1 · Chang et al. · 2018 [cited by applicant]
US 20180188715A1 · Cella et al. · 2018 [cited by applicant]
US 20180190026A1 · Barnett et al. · 2018 [cited by applicant]
US 20180209801A1 · Stentz et al. · 2018 [cited by applicant]
US 20180209803A1 · Rakah · 2018 [cited by applicant]
US 20180238705A1 · O'Herlihy · 2018 [cited by applicant]
US 20180240542A1 · Grimmer · 2018 [cited by applicant]
US 20180278984A1 · Aimone et al. · 2018 [cited by applicant]
US 20180313798A1 · Chokshi et al. · 2018 [cited by applicant]
US 20180342106A1 · Rosado · 2018 [cited by applicant]
US 20180348863A1 · Aimone et al. · 2018 [cited by applicant]
US 20180365904A1 · Holmes · 2018 [cited by applicant]
US 20180374268A1 · Niles · 2018 [cited by applicant]
US 20190019144A1 · Gillen · 2019 [cited by applicant]
US 20190020973A1 · Harish · 2019 [cited by applicant]
US 20190050634A1 · Nerayoff et al. · 2019 [cited by applicant]
US 20190066528A1 · Hwang et al. · 2019 [cited by applicant]
US 20190102946A1 · Spivack et al. · 2019 [cited by applicant]
US 20190108686A1 · Spivack et al. · 2019 [cited by applicant]
US 20190139448A1 · Marshall et al. · 2019 [cited by applicant]
US 20190180862A1 · Wisser et al. · 2019 [cited by applicant]
US 20190186942A1 · Rubin · 2019 [cited by applicant]
US 20190188450A1 · Spivack et al. · 2019 [cited by applicant]
US 20190202448A1 · Pal et al. · 2019 [cited by applicant]
US 20190204110A1 · Dubielzyk · 2019 [cited by applicant]
US 20190228269A1 · Brent et al. · 2019 [cited by applicant]
US 20190236741A1 · Bowman et al. · 2019 [cited by applicant]
US 20190236742A1 · Tomskii et al. · 2019 [cited by applicant]
US 20190251503A1 · Simpson · 2019 [cited by applicant]
US 20190251509A1 · Simpson · 2019 [cited by applicant]
US 20190259008A1 · Lindsey · 2019 [cited by applicant]
US 20190272589A1 · Simpson · 2019 [cited by applicant]
US 20190293438A1 · Simpson · 2019 [cited by applicant]
US 20190304000A1 · Simpson · 2019 [cited by applicant]
US 20190311431A1 · Simpson · 2019 [cited by applicant]
US 20190324989A1 · Borochoff et al. · 2019 [cited by applicant]
US 20190325541A1 · Simpson · 2019 [cited by applicant]
US 20190333166A1 · Simpson · 2019 [cited by applicant]
US 20190333181A1 · Simpson · 2019 [cited by applicant]
US 20190342106A1 · Li et al. · 2019 [cited by applicant]
US 20190353499A1 · Stenneth · 2019 [cited by applicant]
US 20200013498A1 · Gelber · 2020 [cited by applicant]
US 20200027096A1 · Cooner · 2020 [cited by applicant]
US 20200047055A1 · Ward · 2020 [cited by applicant]
US 20200098071A1 · Jackson · 2020 [cited by applicant]
US 20200125999A1 · Simpson · 2020 [cited by applicant]
US 20200151816A1 · Simpson · 2020 [cited by applicant]
US 20200156495A1 · Lindup · 2020 [cited by applicant]
US 20200184416A1 · Javaheri · 2020 [cited by applicant]
US 20200219017A1 · Simpson · 2020 [cited by applicant]
US 20200226853A1 · Ahmed et al. · 2020 [cited by applicant]
US 20200317075A1 · Yokoyama et al. · 2020 [cited by applicant]
US 20200389301A1 · Detres et al. · 2020 [cited by applicant]
US 20210004909A1 · Farmer et al. · 2021 [cited by applicant]
US 20210012278A1 · Alon et al. · 2021 [cited by applicant]
US 20210041258A1 · Simpson · 2021 [cited by applicant]
US 20210042835A1 · Simpson · 2021 [cited by applicant]
US 20210065100A1 · Hwang · 2021 [cited by applicant]
US 20210158447A1 · Simpson · 2021 [cited by applicant]
US 20210166317A1 · Simpson · 2021 [cited by applicant]
US 20210248633A1 · Simpson · 2021 [cited by applicant]
US 20210318132A1 · Simpson · 2021 [cited by applicant]
US 20210326872A1 · Robotham · 2021 [cited by applicant]
US 20210379447A1 · Lee · 2021 [cited by applicant]
US 20210382924A1 · Aaltonen et al. · 2021 [cited by applicant]
US 20220020073A1 · Farmer · 2022 [cited by examiner]
US 20220058578A1 · Javaheri · 2022 [cited by applicant]
US 20220068081A1 · Pariseau · 2022 [cited by applicant]
US 20220100731A1 · Tirapu Azpiroz et al. · 2022 [cited by applicant]
US 20220122026A1 · Okabe et al. · 2022 [cited by applicant]
US 20230157579A1 · Sato · 2023 [cited by applicant]
US 20230377409A1 · Rye · 2023 [cited by applicant]
CN 107341968A · 2017 [cited by applicant]
JP 2003177034A · 2001 [cited by applicant]
KR 20170078094A1 · 2015 [cited by applicant]
WO 9508240A2 · 1995 [cited by applicant]
WO 2015059691A1 · 2015 [cited by applicant]
WO 2015161307A1 · 2015 [cited by applicant]
WO 2019134005A1 · 2019 [cited by applicant]
WO 2019183468A1 · 2019 [cited by applicant]
WO 2021163675A1 · 2021 [cited by applicant]
“Impact of aircraft size and seat availability on airlines demand and market share in duopoly markets” Published by Elsevier (Year: 2005). [cited by examiner]
Papa, U., & Del Core, G., “Design of Sonar Sensor Model for Safe Landing of an UAV,” IEEE Metrology for Aerospace, 2015, pp. 346-350. [cited by applicant]
Li, Jundong, et al., “Multi-network Embedding”, pp. 1-9, 2018. [cited by applicant]
Speediance, All-in-One Smart Home Gym; retrieved from internet: https://www.amazon.com/Speediance-Equipment-Resistance-Training-Machine-Works/dp/B0C4KF7844/?th=1; May 8, 2023; p. 1. [cited by applicant]
Freebeat, Smart Exercise Bike; retrieved from internet: https://www.amazon.com/Resistance-Cushioned-Detection-Altorithm-Instructors/dp/B0BZKKZ6B3/?th=1; Mar. 3, 2023; p. 1. [cited by applicant]
The Wayback Machine, Interest Rate Swaps, https://web.archive.org/web/20171006212154/https://global.pimco.com/en/gbl/resources/education/understanding-interest-rate-swaps, 2016, pp. 1-7. [cited by applicant]
Freight Derivatives—a Vital Tool For YOur Business, https://www.reedsmith.com/-/media/files/perspectives/2007/02/ freight-derivatives—a-vital-tool-for-your-business/files/freight-derivatives—a vital-tool-for-your-busine… [cited by applicant]
Barry, Kieth, App lets drivers auction public parking spaces, Wired, Aug. 11, 2011, pp. 1-4. [cited by applicant]
Jiang, Landu, et al., Sun Chase: Energy-Efficient Route Planning for solar-powered Evs, IEEE 37th international conference on distrubuted computing systems, 2017, pp. 1-11. [cited by applicant]
Netlingo, https://web.archive.org/web/20170122184857/https://www.netlingo.com/word/electronic-exchange.php,dated Oct. 22, 2017. [cited by applicant]
Laseter, Tim, “B2B benchmark: The State of Electronic Exchanges”, Tech & Innovation, dated Oct. 1, 2001. [cited by applicant]
Directed Graph, https://en.wikipedia.org/wiki/Directed_graph, pp. 1-6, 2022. [cited by applicant]
About IBM Food Trust, https://www.IBM.com/downloads/cas/E9DBNDJG, pp. 1-17, 2019. [cited by applicant]
IBM Blockchain Transparent Supply, https://www.ibm.com/downloads/cas/BKQDKOM2, pp. 1-14, Aug. 2020. [cited by applicant]
Radocchia, Samantha, 3 Innovative Ways Blockchain Will Build Trust In The Food Industry, https://www.forbes.com/sites/samantharadocchia/2018/04/26/3-innovative-ways-blockchain-will-build-trust-in-the-food-industry/?sh=6… [cited by applicant]
Change the World, https://fortune.com/change-the-world/2019/IBM/, Fortune Media IP Limited, pp. 1-5, 2022. [cited by applicant]
IBM Food Trust, https://www.constellationr.com/node/17601/vote/application/view/588, Constellation Research Inc., pp. 1-4, 2010-2022. [cited by applicant]
Dey, Somdip, et al., FoodSQRBlock: Digitizing Food Production and the Supply Chain with Blockchain and QR Code in the Cloud, https://www.mdpi.com/2071-1050/13/6/3486/htm, MDPI, pp. 1-27, Mar. 22, 2021. [cited by applicant]
Ramasubramanian, Vasant, “Quadrasense: Immersive UAV-based cross-reality environmental sensor networks,” phD diss., Massachusetts Institute of Technology, pp. 1-75, 2015. [cited by applicant]
Wyzant, https://web.archive.org/web/20190327185429/https://www.wyzant.com/hotitworks/students,Wyzant tutoring, pp. 1-13 , Mar. 27, 2019. [cited by applicant]
PCT International Search Report and Written Opinion; PCT/US2021/065855; Mar. 2, 20229. [cited by applicant]
PCT International Search Report and Written Opinion; PCT/US2022/012717; Mar. 3, 20220. [cited by applicant]
Zhao, et al., Incentives in Ridesharing with Deficit Control, Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2014), May 5-9, 2014, pp. 1021-1028. [cited by applicant]
Peters, et al.; Student Support Services for Online Learning Re-Imagined and Re-Invigorated: Then, Now and What's To Come; Contact North | Contact Nord; Sep. 2017. [cited by applicant]
PCT International Search Report and Written Opinion; PCT/US2022/027077; Nov. 1, 2022. [cited by applicant]
Wei, et al. “impact of aircraft size and seat availability on airlines demand and market share in duopoly markets” Published by Elsevier, 2005, pp. 315-327. [cited by applicant]
PCT International Search Report and Written Opinion; PCT/US2022/052969; Mar. 2, 20231. [cited by applicant]
Little, T.D., et al., On the Joys of Missing Data, Journal of pediatric psychology, 2014, pp. 151-162. [cited by applicant]
Honaker, J., et al., What to do About Missing Values in Time-Series Cross-Section Data, American Journal of Political Science, Sep. 6, 2008, pp. 561-581. [cited by applicant]
Westerhoff, Market Depth and Price Dynamics: A Note, University of Osnabrueck, Department of Economics Rolandstrasse 8, D-49069 Osnabrueck, German, Mar. 30, 2004, pp. 1-8. [cited by applicant]
PCT International Search Report and Written Opinion; PCTUS2022/051998; Mar. 8, 2023. [cited by applicant]
EP23153137.7 European Search Report, May 24, 2023, pp. 1-10. [cited by applicant]
EP20787830.7 European Search Report, May 12, 2023, pp. 1-10. [cited by applicant]
Zheyong, Bian, et al., “Planning the Ridesharing Route for the First-Mile Service Linking to Railway Passenger Transportation,” Joint Rail Conference, Apr. 2017, pp. 1-11. [cited by applicant]
EP23168879.7 European Search Report, Jul. 5, 2023, pp. 1-13. [cited by applicant]
Bortolini, et al.; “Fresh food sustainable distribution: cost, delivery time and carbon footprint three-objective optimization,” 2016, pp. 1-12. [cited by applicant]
Yu, Haicong et al.; “A Multi-Modal Route Planning Approach with an Improved Genetic Algorithm”, The International Archives of the Photogrammetry, Remote Sensing and Spaital Information Sciences, vol. 38, Part 1, 2010. [cited by applicant]
“Node Influence Metric”, Wikipedia, Nov. 6, 2020, pp. 1-5. [cited by applicant]
EP21916571 European Search Report, May 29, 2024, pp. 1-9. [cited by applicant]
Aratani, Lori, “This app wants to reward you for smart commuting choices,” The Washington Post, Aug. 18, 2018, pp. 1-3. [cited by applicant]
Fulldomepro, VR Aquatic Simulator with a Dome, Pub. Sep. 3, 2018, https://www.youtube.com/watch?v=k_53G5DksjQ, pp. 1-2. [cited by applicant]
Randomoneh, Dome Screens & Displays, Pub. Oct. 12, 2012, https:// hardforum.com/threads/dome-screens-displays, p. 1. [cited by applicant]
Borgobello, Bridget, TOOB Personal Dome Screen Revamped, Pub. Jan. 12, 2012, https:// newatlas.com/toob-earth-personal-dome-screen/21082, p. 1. [cited by applicant]
Asghari, et al; “Price-aware Real-time Ride-sharing at Scale-An Auction-based Approach”, Oct. 31, 2016, 9 SIGSPACIAL'16: Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Informat… [cited by applicant]
EP22740218.7 European Search Report, Nov. 12, 2024, pp. 1-29. [cited by applicant]
Karbassi, et al.; “Vehicle Route Prediction and Time of Arrival Estimation Techniques for Improved Transportation System Management” Published by IEEE; 2003, pp. 511-516. [cited by applicant]
EP23153137.7 Exam Report (Communication pursuant to Article 94(3) EPC), Mar. 5, 2025, pp. 1-11. [cited by applicant]
Chang, et al., “Estimating Real-Time Traffic Carbon Dioxide Emissions Based on Intelligent Transportation System Technologies,” IEEE Mar. 1, 2013, vol. 14, pp. 1-11. [cited by applicant]
Valdes, J. J., et al.; (Sep. 2007). Virtual reality high dimensional objective spaces for multi-objective optimization: An improved representation. In 2007 IEEE Congress on Evolutionary Computation (pp. 4191-4198). IEEE… [cited by applicant]
Valdes, J. J., et al.; (2007). Multi-objective evolutionary optimization for constructing neural networks for virtual reality visual data mining: Application to geophysical prospecting. Neural networks, 20(4), pp. 498-5… [cited by applicant]
Zhao, et al., Deshpande, P. M., Naughton, J. F., & Shukla, A (Jun. 1998). Simultaneous optimization and evaluation of multiple dimensional queries. In Proceedings of the 1998 ACM SIGMOD international conference on Manag… [cited by applicant]
A. Lazarus et al., Multimodal Behavior Therapy. Berlin, Germany: Springer, 1976. [cited by applicant]
B. P. Yuhas, M. H. Goldstein, and T. J. Sejnowski, “Integration of acoustic and visual speech signals using neural networks,” IEEE Commun. Mag., vol. 27, No. 11, pp. 65-71, Nov. 1989. [cited by applicant]
W. Guo, J. Wang, and S. Wang, “Deep multimodal representation learning: A survey,” IEEE Access, vol. 7, pp. 63373-63394, 2019. [cited by applicant]
L. Wu, S. L. Oviatt, and P. R. Cohen, “Multimodal integration-a statistical view,” IEEE Trans. Multimedia, vol. 1, No. 4, pp. 334-341, Dec. 1999. [cited by applicant]
J. Shang, T. Ma, C. Xiao, and J. Sun, “Pre-training of graph augmented transformers for medication recommendation,” 2019, arXiv: 1906.00346. [cited by applicant]
K. Gavrilyuk, R. Sanford, M. Javan, and C. G. Snoek, “Actor transformers for group activity recognition,” inProc. IEEE Conf. Comput. Vis. Pattern Recognit., 2020, pp. 836-845. [cited by applicant]
Beltagy, M. E. Peters, and A. Cohan, “Longformer: The long-document transformer,” 2020, arXiv:2004.05150. [cited by applicant]
D. Shin, Z. Ren, E. B. Sudderth, and C. C. Fowlkes, “3D scene reconstruction with multi-layer depth and epipolar transformers,” in Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 2172-2182. [cited by applicant]
P. Xu et al., “SketchMate: Deep hashing for million-scale human sketch retrieval,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 8090-8098. [cited by applicant]
K. Gupta, J. Lazarow, A. Achille, L. Davis, V. Mahadevan, and A. Shrivastava, “LayoutTransformer: Layout generation and completion with self-attention,” 2020, arXiv:2006.14615. [cited by applicant]
T. Hastie, R. Tibshirani, J. H. Friedman, and J. H. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, vol. 2. Berlin, Germany: Springer, 2009. [cited by applicant]
C. Zhang, Z. Yang, X. He, and L. Deng, “Multimodal intelligence: Representation learning, information fusion, and applications,” IEEE J. Sel. Topics Signal Process., vol. 14, No. 3, pp. 478-493, Mar. 2020. [cited by applicant]
C. Sun, A. Myers, C. Vondrick, K. Murphy, and C. Schmid, “VideoBERT: A joint model for video and language representation learning,” in Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 7463-7472. [cited by applicant]
N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in Proc. Eur. Conf. Comput. Vis., 2020, pp. 213-229. [cited by applicant]
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pretraining of deep bidirectional transformers for language understanding,” 2018, arXiv: 1810.04805. [cited by applicant]
Vaswani et al., “Attention is all you need,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2017, pp. 5998-6008. [cited by applicant]
T. Baltruaitis, C. Ahuja, and L.-P. Morency, “Multimodal machine learning: A survey and taxonomy.” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, No. 2, pp. 423-443, Feb. 2019. [cited by applicant]
Cai, et al.; Incorporating Visual Information in Audio Based Self-Supervised Speaker Recognition; IEEE/ACM Transactions on Audio, Speech and Language Processing; vol. 30; pp. 1422-1435; Mar. 24, 2022. [cited by applicant]
Saulle, Rosella, et al., “Cost and Cost-Effectiveness of the Mediterranean Diet: Results of a Systematic Review,” www.mdpi.com/journal/nutrients, Nutrients 2013, 5, pp. 4566-4586. [cited by applicant]
Garamvolgyi et al.; Towards_Model-Driven_Engineering_of_Smart_Contracts; IEEE/IFIP; pp. 134-139; 2018. [cited by applicant]
Khan et al; “A Distri buted-Ledger Consortium Model for Collaborative_Innovation” ; IEEE pp. 29-37; 2017. [cited by applicant]
Meiklejohn S.; Top Ten Obstacles along Distributed Ledgers Path to Adoption; UCL; pp. 13-19; 2017. [cited by applicant]
Muttavarapu et al.; Distributed_Ledger_for_Spammers_Resume; IEEE; 9 pages, 2018. [cited by applicant]
Nikonowicz, et al., “Virtual Power Plants”, Published by Open Acess Journal, 2012, pp. 135-149. [cited by applicant]
S. Pramanik, P. Agrawal, and A. Hussain, “OmniNet: A unified architecture for multi-modal multi-task learning,” Jul. 3, 2020 v2 , arXiv:1907.07804. [cited by applicant]
R. Akula, S. Gella, Y. Al-Onaizan, S.-C. Zhu, and S. Reddy, “Words aren't enough, their order matters: On the robustness of grounding visual referring expressions,” 2020, arXiv:2005.01655. [cited by applicant]
R. Child, S. Gray, A. Radford, and I. Sutskever, “Generating long sequences with sparse transformers,” 2019, arXiv:1904.10509. [cited by applicant]
Y. Xian, C. H. Lampert, B. Schiele, and Z. Akata, “Zero-shot learning—A comprehensive evaluation of the good, the bad and the ugly,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, No. 9, pp. 2251-2265, Sep. 2019. [cited by applicant]
Owens and A. A. Efros, “Audio-visual scene analysis with selfsupervised multisensory features,” inProc. Eur. Conf. Comput. Vis., 2018, pp. 639-658. [cited by applicant]
T. Chen and R. R. Rao, “Audio-visual integration in multimodal communication,” Proc. IEEE, vol. 86, No. 5, pp. 837-852, May 1998. [cited by applicant]
N. Li, S. Liu, Y. Liu, S. Zhao, and M. Liu, “Neural speech synthesis with transformer network,” in Proc. AAAI Conf. Artif. Intell., 2019, pp. 6706-6713. [cited by applicant]
M. Chen, Y. Li, Z. Zhang, and S. Huang, “TVT: Two-view transformer network for video captioning,” in Proc. 10th Asian Conf. Mach. Learn., 2018, pp. 847-862. [cited by applicant]
X. Lin, C. Ding, J. Zeng, and D. Tao, “GPS-Net: Graph property sensing network for scene graph generation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2020, pp. 3743-3752. [cited by applicant]
W. Hao, C. Li, X. Li, L. Carin, and J. Gao, “Towards learning a generic agent for vision-and-language navigation via pre-training,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2020, p. 13134-13143. [cited by applicant]
S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards realtime object detection with region proposal networks,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2015, pp. 91-99. [cited by applicant]
Y.-H. H. Tsai, S. Bai, P. P. Liang, J. Z. Kolter, L.-P. Morency, and R. Salakhutdinov, “Multimodal transformer for unaligned multimodal language sequences,” in Proc. Conf. Assoc. Comput. Linguistics, 2019, pp. 6558-6569. [cited by applicant]
J. Lin, A. Yang, Y. Zhang, J. Liu, J. Zhou, and H. Yang, “InterBERT: Vision-and-language interaction for multi-modal pretraining,” 2020, arXiv:2003.13198. [cited by applicant]
D. Tran, H.Wang, L. Torresani, J. Ray, Y. LeCun, and M. Paluri, “A closer look at spatiotemporal convolutions for action recognition,” inProc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 6450-6459. [cited by applicant]
S. Xie, C. Sun, J. Huang, Z. Tu, and K. Murphy, “Rethinking spatiotemporal feature learning for video understanding,” 2017, arXiv:1712.04851. [cited by applicant]
L. Zhou, Y. Zhou, J. J. Corso, R. Socher, and C. Xiong, “End-to-end dense video captioning with masked transformer,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 8739-8748. [cited by applicant]
X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 7794-7803. [cited by applicant]
J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” 2016, arXiv: 1607.06450. [cited by applicant]
S. loffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in Proc. Int. Conf. Mach. Learn., 2015, pp. 448-456. [cited by applicant]
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770-778. [cited by applicant]
J. Malmaud, J. Huang, V. Rathod, N. Johnston, A. Rabinovich, and K. Murphy, “What's Cookin'? Interpreting cooking videos using text, speech and vision,” 2015, arXiv: 1503.01558. [cited by applicant]
L. Zhou, C. Xu, and J. J. Corso, “Towards automatic learning of procedures from web instructional videos,” in Proc. AAAI Conf. Artif. Intell., 2018, pp. 7590-7598. [cited by applicant]
D. Kiela et al., The hateful memes challenge: Detecting hate speech in multimodal memes, 2020, arXiv:2005.04790. [cited by applicant]
Miech, D. Zhukov, J.-B. Alayrac, M. Tapaswi, I. Laptev, and J. Sivic, “HowTo100M: Learning a text-video embedding by watching hundred million narrated video clips,” in Proc. IEEE Int. Conf. Comput. Vis., 2019, pp. 2630-… [cited by applicant]
Das et al., “Visual dialog,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 1080-1089. [cited by applicant]
R. Krishna, K. Hata, F. Ren, L. Fei-Fei, and J. C. Niebles, “Densecaptioning events in videos,” in Proc. IEEE Int. Conf. Comput. Vis., 2017, pp. 706-715. [cited by applicant]
V. Ordonez, G. Kulkarni, and T. Berg, “Im2Text: Describing images using 1 million captioned photographs,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2011, pp. 1143-1151. [cited by applicant]
R. Krishna et al., “Visual genome: Connecting language and vision using crowdsourced dense image annotations,” Int. J. Comput. Vis., vol. 123, pp. 32-73, 2017. [cited by applicant]
S. Antol et al., “VQA: Visual question answering,” in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 2425-2433. [cited by applicant]
T .-Y. Lin et al., “Microsoft COCO: Common objects in context,” in Proc. Eur. Conf. Comput. Vis., 2014, pp. 740-755. [cited by applicant]
P. Sharma, N. Ding, S. Goodman, and R. Soricut, “Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning,” in Proc. Conf. Assoc. Comput. Linguistics, 2018, pp. 2556-2565. [cited by applicant]
H. Luo et al., “UniVL: A unified video and language pre-training model for multimodal understanding and generation,” 2020, arXiv:2002.06353. [cited by applicant]
D. Qi, L. Su, J. Song, E. Cui, T. Bharti, and A. Sacheti, “ImageBERT: Cross-modal pre-training with large-scale weak-supervised image-text data,” 2020, arXiv:2001.07966. [cited by applicant]
Z. Huang, Z. Zeng, B. Liu, D. Fu, and J. Fu, “Pixel-BERT: Aligning image pixels with text by deep multi-modal transformers,” 2020, arXiv:2004.00849. [cited by applicant]
X. Li et al., “Oscar: Object-semantics aligned pre-training for vision-language tasks,” in Proc. Eur. Conf. Comput. Vis., 2020, pp. 121-137. [cited by applicant]
J. Lu, V. Goswami, M. Rohrbach, D. Parikh, and S. Lee, “12-in-1: Multitask vision and language representation learning,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2020, pp. 10434-10443. [cited by applicant]
L. Zhou, H. Palangi, L. Zhang, H. Hu, J. Corso, and J. Gao, “Unified vision-language pre-training for image captioning and VQA,” in Proc. AAAI Conf. Artif. Intell., 2020, pp. 13041-13049. [cited by applicant]
C. Alberti, J. Ling, M. Collins, and D. Reitter, “Fusion of detected objects in text for visual question answering,” 2019, arXiv:1908.05054. [cited by applicant]
G. Li, N. Duan, Y. Fang, M. Gong, and D. Jiang, “Unicoder-VL: A universal encoder for vision and language by cross-modal pre-training,” in Proc. AAAI Conf. Artif. Intell., 2020, pp. 11336-11344. [cited by applicant]
C. Sun, F. Baradel, K. Murphy, and C. Schmid, “Learning video representations using contrastive bidirectional transformer,” 2019, arXiv: 1906.05743. [cited by applicant]
Y.-C. Chen et al., “UNITER: Universal image-text representation learning,” in Proc. Eur. Conf. Comput. Vis., 2020, pp. 104-120. [cited by applicant]
W. Su et al., “VL-BERT: Pre-training of generic visual-linguistic representations,” 2019, arXiv: 1908.08530. [cited by applicant]
L. H. Li, M. Yatskar, D. Yin, C.-J. Hsieh, and K.-W. Chang, “VisualBERT: A simple and performant baseline for vision and language,” 2019, arXiv: 1908.03557. [cited by applicant]
H. Tan and M. Bansal, “LXMERT: Learning cross-modality encoder representations from transformers,” 2019, arXiv:1908.07490. [cited by applicant]
J. Lu, D. Batra, D. Parikh, and S. Lee, “VILBERT: Pretraining taskagnostic visiolinguistic representations for vision-and-language tasks,” 2019, arXiv: 1908.02265. [cited by applicant]
Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V. Le, “XLNet: Generalized autoregressive pretraining for language understanding,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2019, Art. No. 517. [cited by applicant]
Z. Dai, Z. Yang, Y. Yang, J. Carbonell, Q. V. Le, and R. Salakhutdinov, “Transformer-XL: Attentive language models beyond a fixed-length context,” 2019, arXiv: 1901.02860. [cited by applicant]
Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre-training,” 2018. [Online]. Available: https://s3-US-west-2.amazonaws.com/openai-assets/ research-covers/language… [cited by applicant]
M. Lewis et al., “BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” 2019, arXiv: 1910.13461. [cited by applicant]
Y. Li et al., “BEHRT: Transformer for electronic health records,” Sci. Rep., vol. 10, 2020, Art. No. 7155. [cited by applicant]