New Tool enables copy & paste of real world objects via smartphone
Cyril Diagne has created a fascinating little augmented reality tool from a number of components: with the help of his smartphone you can take a photograph of an object in your environment and then paste it into a picture on a computer screen - so to speak, visual copy and paste of real objects into digital content.
"AR Cut & Paste" consists of several components: one uses machine learning to recognise the object the smartphone is aimed at, digitises it and releases it. The BASNet algorithm is used here. This step is currently still carried out with the help of a separate service on a PC via HTTP, which you have to set up yourself - but in future it can be embedded in the app itself.
In a further step, it is calculated where exactly on the computer screen the virtual object should be inserted. The communication between smartphone and PC runs via a local server on the computer, which receives the object data from the smartphone via the local network and then inserts it in the form of its own layer in Photoshop via remote connection. It is also responsible for calculating the position using the Screenpoint algorithm.
At the moment the tool is still a prototype in the development stage, but anyone with programming skills can try it out for themselves, as the source code including installation instructions is freely available on Github. But all other interested people can hope - because soon there will be an own, easier to install app called AR Copy Paste.
Digital copying of an object still takes about 2.5 seconds and pasting about four, but the respective algorithms can easily be further optimized in the course of further development of the tool. For those who want to know more: in a separate Twitter thread the developer provides information about some questions concerning the tool. We are curious - there are a lot of practical applications for the tool in creative work or communication, for example an integration for cooperation with other programs. The interest in Copy & Paste per AR is enormous: the video has already been viewed about 5 million times.