It promises to leave a deep mark and represent a significant turning point in the history of cinema. For a more democratic cinema, more present among the people, uniting… What was the original cinema, and what it has failed to be for so long.

In this article I want to talk broadly, technically, about the basic idea. And share with you some possible areas of using artificial intelligence to get the most out of it on an, intentionally, small budget.

We can also consider it an update, at a much more advanced stage, of the last article written in February 2023 entitled. How to Make Low Budget Movies with Artificial Intelligence – First Steps. .

The revolutionary impact of Artificial Intelligence in cinema

At one time, special effects and narrative techniques were the magic wands of cinema, but today artificial intelligence is playing a revolutionary card. It is an incredible help, an accomplice that opens the door to unprecedented innovation. We are riding this wave, with AI by our side lending a hand in creating compelling stories, digital characters that feel real, and music that gets right to your heart. But let us not forget the human touch, the real beating heart of each of our creations.

“Artificial intelligence and cinema” is not a token motto, but the beginning of a new chapter in storytelling. We are ready to prove that the future of entertainment is no longer a distant dream: it is here, and it is animated by artificial intelligence.

Luna Dolph, Kyle and China: from virtual to real life

The story, scripted by Gérard Bernasconi , starts with the story of Luna Dolph, Kyle and China; not mere characters, but three-dimensional avatars breathing life into a fascinating and complex virtual world. These avatars are not only protagonists of a story, but symbols of our age. Vivid representations of our progressive immersion in the virtual universe, also known as the metaverse. This concept, once relegated to science fiction, is rapidly taking shape and influencing diverse industries, from video games to social networking platforms, transforming the way we interact, work and have fun.

Virtual Reality and Digital Identities

Our goal is not only to tell a compelling story, but to invite the audience to reflect on the nature of reality in a digital age. These avatars, with their complex interactions and intricate worlds, represent the ongoing fusion of our physical and digital lives. A convergence that is redefining the very meaning of identity, community and belonging.

At the core of our narrative we find Luna Dolph, Kyle and China. They are three three-dimensional avatars whose lives take place in a virtual world of extraordinary beauty. While existing in the digital ether, their story is a bridge to reality, a means of reminding our viewers of the irreplaceable value of human interaction and real life. In an age when digitization has transformed our ways of connecting, our narrative aims to use technology not as a refuge, but as a springboard to rediscover and reinvigorate authentic and tangible sociality.

Technology helps you live better

As Luna, Kyle and China navigate a fascinating metaverse, their experiences and challenges resonate with a clear message: technology, however advanced, is a tool that, when used wisely, can enrich but not replace the warmth and complexity of human connections. Our goal is to bring out the realization that despite the attractiveness of digital, real life takes place off the screen. In the shared laughter, handshakes, hugs and spontaneous moments that make up the fabric of our existence.

With this storytelling, we aspire to inspire viewers to lift their gaze from their devices and re-immerse themselves in the real world, enriching their lives with authentic experiences. Through the exploration of virtual worlds, we want to celebrate and promote the beauty and irreplaceable importance of real life and human sociality.

The film will represent the first outing in the real world for the three main characters.

How we use AI in film production

We are still in the pre-production stage, so from a practical point of view I will keep you updated in the coming months. For now, we have a rough idea: we have selected a compendium of the latest artificial intelligence (AI) technologies that are both affordable and available to all.

The film will be in fulldome, immersive 360×180-degree format. We will basically project it into domes and planetariums. This is a crucial innovation looking to the future as we push more and more toward total immersiveness. And we have a limited budget, let’s say between 10 and 20,000 euros at a glance. The film will last about 40 minutes, and will be about 30 percent in the virtual world of Luna and her friends (entirely recreated in Unreal Engine ), for the remaining 70% in the real world.

Using Luma AI for the cinema of the future

To begin talking about this technological arsenal I mention the following Luma AI, a pioneering solution that completely revolutionizes the generation of three-dimensional environments. Luma AI allows us to reproduce real environments within Unreal Engine, in photorealistic quality and even with an iPhone. Employing advanced technologies such as the Neural Radiance Fields (NeRF) and the brand new Gaussian Splatting (the latter published just three months ago by the University of the Côte d’Azur, France), we can capture the complexity and richness of the real world in digital format, put it into Unreal Engine (including the ability to move freely within the scene) and bring to life scenes previously relegated only to large budgets.

We lower the costs associated with creating detailed settings and complex objects, as well as streamline and speed up the production process. Luma AI not only democratizes access to world-class technologies, but also gives us the tools to experiment at a speed that was previously unimaginable. And it allows us to impart an unprecedented level of vividness and depth to our scenes.

Skybox AI for cinema – Simplified lighting on Unreal Engine

Another crucial tool in our repertoire is Skybox AI by Blockade Labs , for creating immersive skyboxes. Enriching virtual scenes with vital details about lighting and setting.

A skybox in Unreal Engine not only provides visually convincing surroundings (mountains, sky, distant houses, etc…), but also affects the overall lighting of 3D assets within the scene. This is what interests us most in filmmaking: it acts as an ambient light source, reflecting its colors and hues on objects, helping to create a consistent and realistic atmosphere. For example, a skybox depicting a sunset will infuse warm orange and red hues on the scene; while a night skybox will provide a cooler, dimmer light. This process helps integrate 3D assets into the surrounding environment, making the entire visual experience more immersive and coherent.

Aspect that proves essential in creating 360-degree fulldome environments. Every tiny detail is critical to sustaining the illusion of a fully immersive world.

Using Kaiber AI for our film

Kaiber AI is useful and quality, but I have a serious doubt: I don’t know if it works with fulldome video. I haven’t had time to experiment with it yet, but it will possibly help smooth out the scenes so that the assets can be merged even better. It works through AI directly on the final video files.

Artificial intelligence and cinema: Reverie AI

In anticipation of the launch of Reverie AI , we are stimulated by its promises to create virtual worlds for Unreal Engine almost by “copying” a pre-existing picture. Its ability to generate scenery that faithfully mimics reality, combined with its potential in color correction of virtual scenes, opens doors to unlimited possibilities in visual storytelling. Reverie AI promises to be an excellent complement to our workflow, improving visual consistency and ensuring smooth and convincing transitions between the virtual and real worlds.

Move.ai, cheap and working mocap

Another “smart” tool we plan to use for our film will be Move.ai, specifically the inexpensive service Move One (on launch offer at $15 a month, then expected to cost $30). With a simple iPhone app, we will be able to create simple, ready-made custom animations without too many fixes or clean-ups. This reduces time and cost, allowing you not to be limited by the animations already available in services such as Mixamo , ActorCore or the MoCap Market by Noitom.

Here a very quick video test:

Clearly, they also have the more expensive professional service that allows up to 8 rooms to be used. But we do not count on using it for this first production unless there is a very real need.

Speech-to-speech for film dubbing… Is it possible?

In our production, innovation does not stop with the creation of the digital world; in fact, we also want to take advantage of the ongoing technological revolution in traditional aspects of filmmaking, such as dubbing.

The voice of Luna and the other 3D avatars, must be consistent (not to say identical) in all media, both in the virtual and real worlds. We face a complex challenge: maintaining the same voice across multiple platforms, including social media and especially real-time streaming, without tying ourselves to a single voice actor. This allows us to have flexibility in storytelling and to adapt to various formats without depending on the availability of a specific actor.

The idea is to replicate the voices of some real actors and associate them with avatars (Luna, Kyle, and China for starters), then transforming any actor’s voice, in real time, into Luna’s, to use it for both dubbing movies and social content (as much in the original language as in translations), and for the live streaming superimposing these entries on the animated avatar in motion capture.

From the excellent but expensive Respeecher, to the ambiguous Voicemod

We explored options such as Respeecher , an advanced speech-to-speech conversion tool, but the costs for real-time are prohibitive. We’re talking about 1,000 or 2,000 a month for a few hours of use. Voicemod presents itself as a cheaper solution because of its AI voices, although there are conflicting rumors about its reliability (some even consider it to be malware or cryptojacking…). And there remains the problem of not having the rights to the voice, which they own, which will certainly prove to be a problem in the future. I do not yet know the cost for real-time conversion of Resemble.ai , which I have used in the past for the much cheaper text-to-speech, and of Veritone Voice .

Another tool that I have not been able to test is Voidol 3 , at a cost of about $300. I couldn’t find a demo version, but I admit I didn’t even try that hard to request one. It is one of several Oriental software created because of the typical Japanese passion for the anime world. But actually adaptable to our purpose, as we will see in a moment with another Japanese.

MetaVoice Live, Mangio RVC Fork and W-Okada Voice Changer, free and open-source

The final solutions, after much research, are. MetaVoice Live and the Voice Changer from W-Okada. Both open source, which reassures us that we can base the “future life” of the characters on these services, and both free. I cannot fail to mention the YouTube channel AI Tools Search which has been most useful to me. Among the most interesting videos in this area, certainly this one:

I particularly like MetaVoice: it is under heated development and has a cloud version for non-real-time conversions that provides greater quality and flexibility. This one is not free, but the cost between $10 and $25 per month can all in all be dealt with.

In contrast, W-Okada ‘s Voice Changer has many independent developers developing solutions compatible with it, such as GitHub user Mangio621 who created the Mangio RVC Fork , a software with Web interface that can transform the voice of a real actor into that of the chosen voice model. Many of these models, especially famous people, are already available on sites such as Voice-Models.com in “.pth” format. A classic format used in machine learning), but the best thing is that with the same Mangio RVC we can perform training, or training, of a custom voice. All locally thanks to a good video card. And therefore clearly always available, and for free.

I will do a specific article on this shortly, however. So you will follow me as I do some interesting tests.

Generating video with artificial intelligence

One use of AI in film production may be text-to-video, or video-to-video. But how useful can generating videos with artificial intelligence be? I mean right from scratch, describing to the AI in text (or with a very simplified video reference) what you want to achieve. Somewhat the future, to date tools like Runway Gen-1 , Genmo AI , o Moonvalley AI are little more than experiments. Useful at some junctures, but far from the quality and realism needed for a film production.

Evidently, we will have to work for a few more years to get our films

Canon EOS R5C and Dual Fisheye lens for 3D fulldome video

What about live action filming? In our journey, we are trying our hand at a very exciting combination of equipment. We always keep the goal in mind: little expense, much return. I had originally planned to shoot everything in full 360 degrees, also to make it already compatible in case of future virtual reality porting. But the cost became prohibitive, both for the camera (a professional camera like the Insta 360 Titan , which by the way has not been updated for years, costs more than 17,000 euros), and for the difficulties of starting from the very beginning with such a wide field.

So the idea is to produce the right video for the dome, in 360×180 degrees (basically half the sphere). And the lens Canon RF 5.2mm F2.8 L Dual Fisheye , paired with the Canon EOS R5C , should prove to be a winning choice. This setup not only provides immersive images, but also allows us to experiment with stereoscopic shots, which add an extra level of depth and realism.

The Canon EOS R5C camera, with its dynamic range between 11 and 12 stops and 8K sensor, offers excellent value for money. This is an important consideration for us, as we are trying to maintain a balanced budget without sacrificing quality. We will also consider whether to rent it … From Adcom , lens and camera are offered at about 200 euros daily.

To be honest, we can’t fully exploit the potential of stereoscopy yet because of projection, but it is definitely something we would like to explore in the future. It is one of those things that looks really cool and could add a special touch to our project. Have you ever projected stereoscopic fulldomes or do you have any suggestions on how we could integrate it into our work? I would be happy to hear your thoughts and ideas.

And if it will be lacking in anything compared to more emblazoned rooms…. Again, we count on leveraging AI for video quality improvement. But research, in the field, still needs to be done.

Conclusions

In the end, our choice of digital tools and equipment reflects a desire not to compromise on quality while keeping an eye on the budget.

In short, we are creating something beyond traditional cinema. Thanks to artificial intelligence and cutting-edge technology, the “cinema of the future” is no longer a dream. It is real, we want to see it happen. And with the genius of Gérard Bernasconi to the screenplay (who also gave us a great technical contribution), the precision of Michela Sette in the role of VFX Supervisor and the creativity of Michele Pelosio as director, we are forging a revolutionary cinematic experience. Get ready, because we are about to take you to a world where cinema and reality merge into a transcendental experience

L'articolo AI in Film Production: How to Produce an Innovative Low-Budget Film è apparso primo per Future of Cinema - by Dario Riccio.

I got the idea, albeit unintentionally, from Nicolas Perrier from the University of Lyon in France, with one of his posts on LinkedIn. Perrier is a skilled expert in innovation in augmented and virtual reality, and the post in question is about Plask; one of the many tools for creating 3D animations starting from a simple video. In practice, it’s Motion Capture without expensive and complex equipment. This technology may be of particular interest to us to produce animated videos at a fraction of the current cost, and even with actors remotely.

I’m also getting additional help, not just from Nicolas himself with his countless posts, but also from the website Futurepedia.io. It’s an “AI wiki”, featuring a selection of many tools currently available to the general public.

Let’s analyze some of these tools, specifically the ones that are useful in filmmaking. Both for writing and for technical production of videos, as well as for voices. We’ll evaluate the quality of the results, conduct experiments, and learn about their costs.

To better understand how to use them in our low-budget films, I decided to create a short film (with very low expectations, just for technical experimentation) using them as much as possible.

Making videos with artificial intelligence.

Let’s divide the services into three main categories: writing, video, and audio. Starting with the writing, having to have the idea first.

Film writing with artificial intelligence

We need a story. Created by an AI? Let’s see, writing tools are not lacking. And if you don’t agree, you can always argue in the comments.

How does GPT-3 work?

Most public AI writing services are currently based on GPT-3, which has 175 billion machine learning parameters. The alternatives are actually many: BigScience Bloom, a large-scale language that has recently been launched (with the advantage of being open source), or the German Aleph Alpha with its Luminous (with 200 billion parameters).

What are the parameters of an artificial intelligence?

Imagine having a task that requires predicting whether an image contains a cat or not. A machine learning model could be trained on many images labeled as “cat” or “not cat” to learn to recognize the distinctive features of cats.

To do this, the model uses a neural network, which is a set of interconnected nodes. Each node represents a computation performed on the input data. The parameters, or “weights,” are the values assigned to each node that affect the strength of the connection between nodes.

When the model sees a new image, these weights are used to perform a series of calculations that ultimately produce a prediction, such as “cat” or “not cat”. The number of weights used in the model is the number of parameters.

In summary, parameters are like “ajuste values” that influence how a machine learning model processes data to make a prediction, and are modified during training to improve the model’s performance.

A model with many parameters will have more opportunities to adjust its calculations to fit the training data, but it may also be more prone to overfitting, meaning an excessive adherence to the training data and a poor ability to generalize.

Other competitors of ChatGPT.

Other competitors are Nvidia with the Megatron NGL (huge, but not available to the public ), Google with their GLaM, LaMDA and PaLM (yes, they don’t leave anything out), OPT by Meta, AlexaTM by Amazon or various Chinese ones like Alibaba and its BaGuaLu.

To conclude, I mention GPT-J, another open source alternative to GPT-3.

Let’s quickly understand how GPT-3 works, using a rephrased response from the AI itself:

GPT-3 (Generative Pretrained Transformer 3) is a state-of-the-art language model developed by OpenAI. Utilizes a deep learning technique called “transformers“to generate human-like text based on a given input.

Here’s how GPT-3 works:

1. The model is trained on a large set of texts generated by humans, such as books, articles, and websites. This training process allows the model to learn patterns and structures of natural language.
2. When it receives an input, the model processes it using its layers of neural networks. When it receives an input, the model processes it using its layers of neural networks. Each layer performs a different task, such as analyzing the syntax, semantics, and context of the input.
3. The model then generates a response based on the input and its understanding of natural language. It does so by predicting the next words in the sentence, using the patterns and structures learned during training.
4. The model repeats this process several times, generating multiple responses to the input. These responses are then ranked and evaluated by the model, and the highest-scoring response is selected as the final output.

GPT-3 is known for its ability to generate text that is often difficult to distinguish from text written by a human being. It is used in various applications, such as language translation, text summarization, and text generation.

ChatGPT can tell stories for screenplay.

Let’s start directly with OpenAI’s product: ChatGPT. It is perhaps the tool of the moment; the first one to have made a leap towards the general public. It’s almost concerning how accurate and “real” it can be in conversations, but it has also been discussed too much and everywhere in the last few weeks.

I signed up for the OpenAI website and opened the chat to try talking to GPT-3 (I can’t imagine what GPT-4 will do next) by asking it to write me a story with some dialogues for a short digital graphic film. And well, it simply made it for me.

The story written by ChatGPT

This is the story as written by ChatGPT, with no modifications, translated into Italian (you can read the original in the English version of this article).

Let us consider this first story. Of course, he didn’t make it easy for me…. He could create a story in the middle of a random city; in contrast, the jungle will be more complex to recreate. But I have to be in it, that’s the game.

The same story, in a short version

Oh, I also asked for a shorter version. He gave me a summary. Then I specified that I wanted a short version but with dialogue, and he simply did it. I am reporting it to you for completeness:

ChatGPT automatically writes the script

I was considering collecting it into a screenplay, when I thought, “Why not ask him?” Said and done. He rewrote the story for me already ready for film:

Apart from Inside Jungle, it is otherwise a good starting point. But some form errors jumped out at Andrea Rocci, an “human” screenwriter, albeit without an in-depth analysis (we were chatting over a beer).

In fact, the script speaks in pictures, and phrases such as “discovering more and more treasures,” or “rich beyond expectation” are not at all clear. Which treasures? Statues, coins, anything else? And, what do you mean by rich? Are they filled with gold on them? Are they dressed flamboyantly? Everyone with the latest iPhone and the keys to a Ferrari?

Not to mention the lack of descriptions of environments. The jungle itself is left to the fullest imagination of the director, or set designers (or 3D artists, whatever).

However, we must make a virtue of necessity; we will leave any choice at the discretion of the director (if he existed, at least…). We will try to find a good one on character.ai, perhaps. In fact, try it out and talk to artificial “characters.” Even Albert Einstein is there!

Prices

Here it’s simple: it doesn’t cost anything basically. A $20/month version is starting to be marketed in some countries, which removes some limitations (mainly due to the computing power needed to handle the millions of requests coming into OpenAI each day).

Alternatives to ChatGPT

At present GPT-3 is hard to beat… While waiting for the most emblazoned candidates to come out (Google Bard soon), I asked ChatGPT itself about its competitors. He pointed me to ScriptBuddy, WriterDuet and AI Screenwriter to begin with. Asking for more, Plotbot, Amazon Storywriter, and InkTip Script Listing. Okay, I thought that was enough… Except that the answer is actually a partial lie. Here we see the current limitations of this artificial intelligence, in part (but not only) due to the fact that the data it has is up to 2021.

Meanwhile, ScriptBuddy, originally based on GPT-2, has not been maintained for more than 3 years; we exclude it. WriterDuet, on the other hand, is most useful in script arrangement. But he doesn’t write anything himself… AI Screenwriter I don’t think even exists, at least dear old Google doesn’t give me any results.

As for Plotbot, or rather Plot-bot, it actually generates movie plots by exploiting the GPT-2 engine. But, like ScriptBuddy, it has not been maintained for years, and to optimize time, I rule it out. The same is true for Amazon StoryWriter, while InkTip Script Listing is not for generating text.

Dramatron

Dramatron is a new alternative, branded Deepmind, that promises exactly what we are looking for: writing screenplays for film and theater. Unfortunately, it was only released a couple of days around December 9, and by the time I got to try it, it was too late.

It remains an interesting project to keep in mind when it is opened. For the time being, George Mike, author of this YouTube video, does not like it.

His comment? “…overall, it seemed insipid to me. And it didn’t give me much more than the dialogue that you find in all the movies. He clearly grasped the genre, but did not explore it in an interesting way. It did not advance the narrative“.

Rytr

Through Google and Futurepedia I then came to Rytr. A nice tool. Account and login, and I find myself in the work area. He basically asks for language, style and use case of the created text, and he acts. Since there was clearly no “film script,” I tried meanwhile with a tool called a “magic wand.”

Same request made to ChatGPT, however, the response was almost hilarious:

Let’s try something different and more specific… I asked “I need a script of about 2000 words, with dialogue, to make a CG short film. The title is: the lost city”, and set as a use case “story plot.” Prompt based on the previous ChatGPT output. But we are still a long way off, although we have a basic storyline:

Same request, but with the magic command as the use case:

Okay, a useful tool for short texts but still far from total autonomy. We come to prices, although in our case I maintain ChatGPT’s text.

Prices

Rytr has a completely free version, limited to 10000 characters per month. Otherwise it costs $9 per month for 100,000 characters, or $29 per month for unlimited use.

Shortly AI

Another tool is Shortly AI, primarily designed to combat “writer’s block”. Write your text or a sentence, and he continues.

Since I could not ask him to do the story for me, I used the first sentence of the story generated by ChatGPT: Quando il sole sorse all’orizzonte, un piccolo gruppo di avventurieri si mise in viaggio per trovare la città d’oro perduta. Although based on the same language model, the continuation was clearly different.

This is the final text:

Ultimately, a good story but with little meaning. The AI took bits of text from here and there, but without connecting them in an “intelligent” way.

Prices

Shortly AI is certainly useful for adding a sentence or two and overcoming writer’s block, but to let it do its own thing does not seem the best tool.

Nor is its cost negligible. We are talking about $79 per month, or $780 per year. I still used the free trial, so it is only required to register an account.

WriterX

Let’s look at WriterX. It has templates for creating articles, social posts, text conclusions and many others. I expect similar results to Shortly, not being designed for whole stories.

Unfortunately, however, the free trial version does not allow access to all features, and I didn’t feel like paying for a service (even a fairly expensive one) without first trying it out; even having a free alternative that I am partially satisfied with.

Although I am evaluating it for its useful marketing features… Like the bio of my social channels, or any future help in writing blog articles. I also used it to write the meta description of this same article.

Prices

WriterX costs $29 a month in the standard version (basically the trial I had), or $59 a month to have unlimited text and functions. It is available in 25 languages.

Jasper AI

I also wanted to try Jasper AI, which is ultimately a graphical user interface for GPT-3 itself. It is perhaps the most publicized, found everywhere. But I simply haven’t even begun to use it: it forces you to enter your credit card even for the free version, and to verify it takes not a few cents but the entire first month: $29. Unprofessional attitude, so I don’t want to deal with them and wanted my money back immediately.

GPT-J and Writey AI

To get out of the GPT-3 universe, I wanted to try the open source GPT-J via the 6b.eleuther.ai website; however, it always crashed with the message, “Unable to connect to the model. Please try again”. And Writey AI, also well-functioning but too specialized in writing blog articles. Which I recommend you check it out for, if only because of 5 articles a month totally free.

Ultimately, I am tired and any further research seems futile. After all, ChatGPT’s text is valid (if you can call it an “automatic” text), so I would say let’s move on to the technical realization of the short film.

Creating 3D characters

Can an artificial intelligence generate 3D characters?

So we need the characters for our story, but is it really possible to generate them with A.I.? Spoiler: today, not well. There are many promises and some solutions that come close to the result, but it is not yet possible. Let us look at them in brief, as they will be useful in the near future. But then we will go on to figure out how to have the characters in our story now with little money.

PIFuHD

To start, there is PIFuHD, which is already available to the public and promises to create a 3D character from a single photo. It works, but even from the presentation videos, a quality far from acceptable in the cinema is remarked upon.

Google DreamFusion

DreamFusion is one of the first A.I.s that can leverage 2D data from Stable Diffusion (the same data used to create photos using artificial intelligence, which is now widespread), to create 3D models.

Same problem as before: unsatisfactory quality even at first glance.

Microsoft Rodin Diffusion

A 2023 newcomer to Microsoft, Rodin Diffusion is not yet available to the public. It promises to create a realistic 3D avatar from a single photo. Well, judging from the photos released on their site the hair… Those are always the problem!

Nvidia Get3D

You certainly cannot miss the queen of graphics cards, Nvidia, among the 3D tools. And indeed its Get3D is superlative in creating 3D models from information learned from a dataset of 3D models.

The dataset is much more limited, and this is a disadvantage in the variety of 3D objects that can be realized. It is, however, open-source, a definite plus point, although the entire training must be done in one’s own system. This means high-end video cards galore… And they cost a little. Much.

Nvidia Magic3D

How did Nvidia solve the problem of the “limitation”, in terms of quantity, of objects that can be created by Get3D? As the article in The Decoder explains, simply copying Google… And trying to make its new Magic3D (researchers’ paper here) faster and more defined than DreamFusion. This video explains well how it works:

In practice, Get3D has a dataset based on other 3D models. Instead, Magic3D starts with images, themselves generated by A.I., paving the way for virtually infinite combinations.

Does it work well? What is certain is that it is not available to the public, but from the videos and examples on the Web it looks like a very promising technology. Although it still does not reach the necessary quality.

StyleGAN-NADA

For doing Pokemon or other fun things there is StyleGAN-NADA, trained following OpenAI’s CLIP (Contrastive Language-Image Pre-Training) model. It allows you to create images from just a textual description, without the need to see any references and without the need to collect additional training data.

It is also possible to modify existing images to make them similar to those in other domains, for example, using an image of a dog to generate a cat. The same approach can be applied to other generative architectures, opening up interesting possibilities for creating images quickly and accurately.

Pollinations

Among the “next steps,” Pollinations promises to do what we need it to do. From their website, “at the research level, our team is developing technology that allows people to generate 3D objects and avatars with the help of text alone”.

Here again, there is a wait. For now, it still allows you to do interesting things in the photo/video area. Maybe try it, however there is little of use for the purposes of this article.

Text2mesh

Small but interesting, Text2mesh is less of an exercise in style than its predecessors. Here you already have to have the model, but the AI promises to modify it independently; for example, increasing the number of polygons, changing its shape and color even creating the texture from scratch. All based on a text prompt, a written request.

Reminder to put in the diary in case we need it.

Luma AI

Luma AI is an interesting project to scan real objects by recreating them in 3D. The operation is interesting, and the quality of the scans is reasonably good. For props or figures in the background, I consider it more than acceptable, even in production.

It has also recently allowed you to create objects, and thus characters, in 3D from a text prompt. With the classic “imagine” command, already made famous by the MidJourney image generator. But here, again, the quality is not sublime. Good experiments, but definitely not usable for production purposes.

3D characters and objects without artificial intelligence

From all of this we understood only one thing: A.I. as of today, January 2023, still does not allow us to have good 3D models. Since we have to keep the budget low, however, let’s achieve them with the tools already available.

3D characters in our short film

I won’t go into a lot of research here, but I evaluate two 3D character creation tools that I already know: the simple Reallusion Character Creator, and Epic’s fantastic MetaHuman.

We will need 3 characters for our story: Maria, Jake, and Sarah.

MetaHuman Creator

Actually, in the case of MetaHuman there is a change from my past: I used the beta of MetaHuman Creator. It is phenomenal in that it has given me the ability to take advantage of the computing power of Epic’s servers by creating characters in a work break directly with my laptop (which only needs to receive a video stream).

So by requesting “Early Access” with one’s Epic Games account from metahuman.unrealengine.com, we end up with a choice of possible characters.

We imagine that all three are between the ages of 20 and 40, otherwise the script does not tell us much about them.

Jake

I choose to begin by selecting Aoi, as Jake. I don’t know, that beard gives me the idea of “adventurer.”

The software warns that some elements of the character has elements still under development (specifically hair) and therefore only LOD (level of detail) 0 (automatic) and 1 (highest quality) will be displayed. For us it is fine, the destination will be a pre-rendered video clearly at the highest quality and not a real-time video game.

A few changes to the character (shirtless, eye color, more “suitable” shoes and pants), and he is saved. Then we will export it with the Quixel Bridge plugin of Unreal Engine 5.

Maria

Let’s move on to Maria. I asked ChatGPT to come up with its characteristics, and the answer was that it could be a woman around 30 years old. Brown hair, shoulder-length and pulled back into a ponytail. Brown eyes, intense and deep, and of Latin ethnicity, with skin tanned from his outdoor adventures.

Let’s try to realize it. Let’s start from Roux. Let’s make a Blend with Lena, Kendra and Tori who seem suitable to modify her features and thus ethnicity a bit, give her a ponytail, brown eyes, modify the texture of her skin to give her a few more years, eliminate the make-up she would hardly have in the middle of the jungle, modify her clothing and that’s it.

Sarah

Finally Sarah. For ChatGPT he is about 25 years old, with short, wavy blond hair. Blue eyes, lively and inquisitive, around 1.70 meters, slender and muscular indicating an active and sporty person and of Northern European descent, with fair and delicate skin.

We rely on Vivian, various modifications until we make it something like the required. Clearly MetaHuman has many limitations, even more so in this online version. For example, on the body we have practically no possibility to operate, so “muscular” is a feature we will have to renounce, unless we model later. But, first I am not a 3D modeler; and this is a zero-budget project for educational purposes only. Also, for the same reason, it is not the case to waste more time on it than necessary.

Finally let’s leave them there; we’ll create the animations with mannequins and then we’ll retarget with MetaHuman characters directly in Unreal Engine 5.

Conclusions

Let’s limit ourselves here for today; in one of the next articles I will talk specifically about animation and Motion Capture with artificial intelligence (where it will be most useful to us), and then we will continue with environments, voices, music, and whatever else we need to finalize our little project.

I will give myself time to finish slowly, and possibly do other articles before continuing this one. For two reasons: they are elaborate operations, and artificial intelligence is in an explosive phase. An article written today may be old tomorrow. Maybe tonight.

Therefore, since we will need this information much more toward the end of this year for the actual production of a fulldome story, let us keep in mind all the news in the coming months.

As always, thank you for following me, and a hug.

L'articolo How to make low-budget films with Artificial Intelligence – Early stages è apparso primo per Future of Cinema - by Dario Riccio.

In his voice we can hear, without a doubt, a passion for the new way to watch movies, with an immersiveness beyond any level ever achieved. We could say that they have “recreated” the movies in the real world.

Let’s try to understand it a little better with himself. So here is the interview with Andrea Moccia.

Introducing Andrea Moccia

Dario:

Andrea Moccia is a veteran in innovative cinema who has been trying for years to innovate and change a bit the way cinema is enjoyed.

Andrea:

Hi Dario, nice to meet you. I am very glad to meet you.

Dario:

How are you? All right?

Andrea:

It’s all right, thank you. Here in London it is gray as hell. Today there is a fog I have never seen, it feels more like Switzerland than London.

Dario:

Okay, a Londoner telling me about the fog like that is interesting. People say it’s always there, but I’ve been to London a few times actually, and I’ve never caught it with fog….

Andrea:

Fog not too much. The sky is always gray. But fog not too much.

Dario:

Yes, indeed the sky was gray. I clearly remember that. So, tell us a little about you… You’re from Rome. So a pure Italian, from the capital. Moved to London since? What was it like? Why did you go to London? How did you find yourself there?

Andrea:

I moved to London in 2010 for studies. I graduated here, and during my last year of University, I was very lucky to be introduced to the world of Secret Cinema. I went to a couple of their shows, and among them there was one based on The Shawshank Redemption. After going to this show, I was absolutely impressed by this world, this format. And I found a way to start working with them.

At first as a trainee. While I used to study in the mornings for exams, I went to the office in the afternoons. Then I started working with them, first as an assistant and then for the last five or six years as a producer.

Dario:

Okay, I just stop you a moment, just to get a good understanding. First of all, I presume you were studying something related to cinema. What were you studying?

Andrea:

No, I did business administration and management. So not directly related to cinema. However, it is a degree with which I found many affinities as a producer. Let’s also say that I’ve always had an interest in art, in performance. When I was a kid I did some acting, I liked that world. And I found a way to apply my degree to the world of performance and art.

How Secret Cinema is interpreting immersive cinema

Dario:

OK. Because in fact Secret Cinema, even to explain it a little bit to our listeners, what is it? Very broad question.

Andrea:

Of course. So, Secret Cinema is a company that specializes in what we call “immersive cinema”. And what the company does, is basically take these big abandoned buildings, or big spaces, pick a movie and rebuild the whole set and world of the movie inside this building.

Then spectators buy a ticket. The moment they buy a ticket, they are assigned a character. We tell them how to dress and who they are within the world. Never anyone among the main characters. Always characters created by us who could exist in that world.

Dario:

Extras…

Andrea:

That’s right, extras basically. And people come to experience the world of this film. We have covered great classics such as Back to the Future, The Battle of Algiers, Star Wars. Up to much more niche films, for example Park Chan-wook‘s The Handmaiden, or I, Daniel Blake.

The company was founded in 2007. It operates on a global scale at this time. We have performances in the United States, England and China.

England, the ideal country to grow up

Dario:

Basically, however, you do most of the shows in London. Or they are somewhat evenly divided among countries.

Andrea:

Most of the shows we always did in London, yes. Let’s say London is our base, where we grew up. And England has been crucial to that growth, I think.

In terms of bureaucracy, but also as theatrical industry, there is already a base where something like this has been able to grow far more easily than in other countries. For example, in my opinion even if the idea had started in the USA, it would have been more complicated. Even just to access these buildings, which are usually large mansions not built to accommodate thousands of people for an event.

They can be abandoned factories, or warehouses, things like that. So the process of getting permits, the process of turning these buildings into what they are, then assets or spaces for very large events, is very complicated and expensive.

And I think, in the early days, these ingredients could have worked only in England.

Close cooperation with film companies

Dario:

Do you guys have any help, any cooperation from the film companies, or do they not care? Do you just buy the rights to reproduce the film?

Andrea:

Look, it really depends from movie to movie. There are some properties that are perhaps a bit more dormant, with titles that are not being redeveloped or, otherwise while having historical value, are not bringing in much revenue at this time.

At that point maybe the studios contact us and ask if we want to do something with these films; in short, they want to generate new interest. Then, of course, there are other films where we ask for and pay rights to the studios, working with them as licensees.

There is always collaboration at the operational level, though, never just at the financial level. There is always an involvement of “Studios” and creators in our creative process.

Dario:

Okay, so they are interested anyway. It is also a way for the companies to promote the film.

Andrea:

Absolutely. We make many films in years when a sequel is released. For example, Blade Runner, which we did in the year that Blade Runner 2049 was released. Or Star Wars in the year the franchise started again. So yes, many times there are also marketing interests behind it.

A beautiful thing Baz Luhrmann said to Fabien (Fabien Riggall, nda), the person who created Secret Cinema in 2007, after coming to a Moulin Rouge! show, was:

I made the movie, but you created the world.

Baz Luhrmann

And that, I think, is a very appropriate summary for what we do. We create a world of the film.

How to choose films suitable for immersiveness

Dario:

You are currently working with Marvel, and then with Guardians of the Galaxy. For example, who wanted to make this film that is now on the schedule? You asked Marvel or it was Marvel who asked you to develop the world, the marketing.

Andrea:

Guardians of the Galaxy is the last show that I worked on at Secret Cinema. I recently left Secret Cinema.

In reality, it is originated from a much larger collaboration between Disney and Secret Cinema. We have a contract of several years with Disney in which we have access to several of their titles. I mean, who wouldn’t want to work on the Marvel world? Who wouldn’t want to join the Marvel Universe…

Dario:

I guess.

Andrea:

Basically when we choose movies, we don’t always choose them because it’s a great movie or a great story. There are many beautiful films that do not work for a Secret Cinema, precisely because we have to create a world, we have to give viewers the opportunity to disguise themselves, to impersonate someone.

Guardians of the Galaxy, being all set in space with this mix of alien races, we felt it was the most interesting world we could recreate in a physical, real-world setting.

Whereas instead, I don’t know… One of my favorite Marvel movies is the first Iron Man. However, it does not offer many insights to engage viewers. While instead you come as an alien, to watch Guardians of the Galaxy. So, all the people come in makeup and in these beautiful, exaggerated costumes; and part of the beauty of coming to a Secret Cinema is just that, right?

There are people who spend weeks, months, creating costumes for themselves, putting all the photos of the process on Instagram. Many of our fans are also cosplayers, or people who work in the creative industry. It is also an opportunity for them to show off their creative talents in a social, sociable environment.

The typical structure of a Secret Cinema

Dario:

Oh yes, because the Secret Cinemas, practically, how are they structured? Is there the screening of the movie in costume, or you see the movie and then move on to other environments…

Andrea:

In a typical Secret Cinema, we sent you basically an email. The email redirects you to a website. On this site we usually have personality questions, questions that will get you assigned a character. Each character is part of larger groups, and each group has its own specific mission; its own specific path within the show.

Then, when you get to the show, the first two or three hours are like an open world video game, where you can go and do various missions; talk to all the characters. You go to sets, you can get drinks, you can go dancing, there’s music, there’s food… Yes, it’s like a video game where you choose your own adventure.

Dario:

A real video game, in short, rebuilt in reality…

Andrea:

Correct. So this is the beginning. Then, after two or three hours, you watch the movie. And during the film we also reenact many live scenes. So, for example, if you’re in Back to the Future and there’s the DeLorean leaving and going to 1986, there’s also a DeLorean going next to you. Just a real DeLorean passing by you.

Or, when Doc falls off the tower, we had a Doc starting from the clock tower on a zipline, going over the spectators and landing on the other side of the field. Because Back to the Future is a show we did outdoors.

For Star Wars, we had an X-Wing that flew over a thousand viewers, landed and out came Luke Skywalker. So it’s all things like that. These are shows that reach production values that few other events can achieve.

Is VR the future? Or will we get back together again?

Dario:

Of course, the parallel with virtual reality comes to mind. In the sense that you have similar experiences, with the major difference being that you are in a sociable environment, with others, and not alone locked in an headset. And this could also make a big difference.

In your opinion, or even according to people you have talked to (viewers, colleagues…), how will this situation evolve? Will people always want to be with each other?

Andrea:

I, at least, am an absolute fan of shared experiences. For me, there is nothing more beautiful than going to a concert with thousands of other people; or going to a movie or theater.

There is just a feeling that you can’t beat. Unless virtual reality will reach levels we cannot even imagine at this time. I don’t think it will come to replace the live experience.

What we are trying to do is to put some magic back into going to the cinema, right? Fabien, when he started this concept of Secret Cinema he was always thinking about the days when people went to the movies dressed up; when going to the movies was an event, when it was really a celebration.

This is what we want to report. It came into being almost a reaction to these multiplex giants. These multiscreens where the experience of going to the movies has been somewhat “formatted,” and the magic has been lost a bit. So this is what we are looking for. And this, in my opinion, is the challenge that cinema, and other art forms as well, will have to face.

But how do you do it? How do you put this magic back? How do you go about breaking a format that has come to produce this mechanization of entertainment? I don’t know how to explain…

So many seek a new way of conceiving cinema

Dario:

Yes, everything became very standardized. In fact, this is something I am glad to hear you say. Because I also support it, personally and in my blog. In mindset I am like that. I also mentioned you, in the email I sent you, that I follow a lot the studies and life of Douglas Trumbull, the creator, the maker of the special effects of 2001: A Space Odyssey.

Andrea:

But also of Blade Runner. I think he died recently, Douglas Trumbull.

Dario:

He died very recently, this year in February I think. He, however, has spent this life trying to find a solution. Obviously its was not Secret Cinema. It was something less “scenic”, but still perhaps more reproducible. That’s the difference.

Secret Cinema I think is a wonderful thing. I still don’t, I just met him recently, but I will definitely have to participate because it is beautiful as an idea. It is very expensive, though; organizing a single screening will cost impossible sums of money for a small business to handle.

Andrea:

Absolutely. However, there are manners. The shows they are doing now are shows of ten million and up per show. So very, very large productions, with very real economic needs. You have to sell hundreds of thousands of tickets to make these things work.

Skates and social issues: where Secret Cinema started.

But let’s remember where Secret Cinema started, though. Because Secret Cinema started with a screening in an abandoned skateboard park, where they basically projected that Paranoid Park on a sheet. And two hundred people came.

So there are simple ways to do things in an effective way, especially when you have films that maybe deal with serious and difficult issues. At that moment you don’t really want to go and recreate the film, but you want to bring the theme to light more. And so we also do a lot of social things.

For example, we did screenings against censorship when films were censored; then screenings in refugee camps. Cinema is not just entertainment, is it? There is a whole sector of cinema that is about entertainment, fun, and it is beautiful that it exists. But there is also a whole part of cinema that wants to send strong messages.

And that in my opinion also needs to be reinvented. Spaces must be found. I am a huge Marvel fan, but you can’t just go to see Marvel at the cinema. Other kinds of films should be seen as well, because there are beautiful things out there.

Dario:

Yes, and then cinema almost was born for that. Well, apart from the very first screenings, the development of cinema internationally was because of this. To pass messages, at the time perhaps more political than anything else, which then quickly became moral or various messages in the early decades.

Andrea:

But we may have forgotten that a little bit.

The importance of moral awareness in cinema

Dario:

Effectively, yes. Oh God, in part. From one point of view, even just many movies or series on Netflix still try to maintain a moral. Then, currently, even those have become standardized.

In the sense they go by fads. At this time I personally notice feminism or LGBT+ is trending; there is an attempt to raise awareness of these issues. Sometimes there is the topic of animals, meat, vegetarians. However, we often talk about fads that are then forgotten. When, instead, perhaps it should be more consistent.

Andrea:

Yes, it is true. And in my opinion, here again we come back to the issue of commerciality. If people don’t go to the cinema, don’t buy tickets, unfortunately these films will no longer be made. Ways have to be found to bring people back to the very act of going to the cinema.

I have been following Cinema America in Rome for many years, and what they have been doing: bringing cinema to the streets, and various initiatives just to rekindle the passion for film, to go to the cinema and watch something all together. Which I think after two years of pandemic we really have to “have it taught” almost, don’t we? People are perhaps still a little afraid to go out, to socialize and are definitely more cautious. However, there are ways to continue living without taking risks.

Dario:

In past years I have organized various outdoor cinemas around Italy. We have done cinemas in Tuscany, in Montalcino, in Ladispoli, and also in Rome, in various places around the capital, in Abruzzo… I mean, in different places and basically they were all very, very well attended.

Of course, they were free cinemas; they were almost always contributed by municipalities or associations. So there was no ticket clutch. But people liked them; they went there; they spent evenings together and they were all super happy to do something different.

The problem is that, precisely, they were happy to do something different. Instead, it should become the norm. I remember my father, born in 1937 (other times…) telling me about when he was a child, or at least a teenager, and the movie theater was kind of their bar. In the sense that they went to the movies almost every day. It cost very little, and they went there to be together and spend two hours. I mean, it was just different culture from this point of view, but then it evolved.

The death of the high street

Andrea:

And maybe that’s part of the solution too, right? The Internet, new technologies, have changed the way we do everything. For example, here in England we are experiencing a time when there is a lot of conversation around “the death of the high street“, the death of the street with all the stores. And how can we reinvent stores so they don’t die? How can we reinvent the high streets?

Stores are reinventing themselves. Here a clothing store, is now no longer just a clothing store. It is a clothing store with a cafe, with a bookstore. A bit of a gathering place. And perhaps cinemas also need to reinvent themselves and become gathering places. It would be great if, for example, all theaters had a small library on the history of movies…

I mean, in my opinion, you have to detach yourself a little bit from this format of cinema. You go, you get the popcorn, you get the Coke, you sit down, you just leave. If cinema is turned into an experience, if cinema is turned into a reason to go out, to socialize, in my opinion it will find its place again.

Dario:

Definitely. Experiences also and especially not reproducible at home. Because the problem with current cinema is that it is reproducible almost entirely at home, with a video projector or a simple TV. You can eat popcorns, you can drink Coca-Cola or beer. I mean, it lacks perhaps that extra reason that makes you say: I can only do this thing there. And we all are lazy, basically, as the human race.

Andrea (smiling):

Me first.

Dario:

Me too, don’t believe. And so, rightly you stay home. But yes, there should be one more reason.

Multi-sensory cinema: yes, but without distractions

Let’s talk about one last thing, then if you want to add more, otherwise we’ll end since we’ve already been half an hour together. I wanted to ask you, what do you think about using more than two senses at the cinema? Have you ever experimented with combining smells, possibly flavors, so a combination with foods or other things but always related to the film at the cinema? Also touch, albeit more complex (although in Secret Cinema there is actually touch, you can touch things of the film as well).

Andrea:

It is a trend that is going a lot, for example in Asian countries. There are some big companies that have started to make these really big 4D cinemas where they include “smell,” water splash, something like that. Or there are other cinemas, especially here, where for example during a screening they bring you food or drink.

So there are already things that unite them. For me, the question one must always ask himself is: how do these things go to enhance, to improve, the viewers’ experience? If these things are done with the intention that maybe a certain smell can improve my understanding of the film, of what I’m going to see, offering a new perspective, then I’m absolutely pro. If these things are just being done to do something new, then I think they can occasionally have the opposite effect. Becoming distractions rather than a magnifying glass.

Dario:

Okay, you say that they could become something that makes you lose the sense, the lesson. It goes back to the argument we were making earlier.

Andrea:

Correct. However, I am always “pro” experimenting, I am always “pro” trying new things. New things need to be tried, and they also go in this direction. Absolutely. However, as long as the intention is there. Now I can’t give you any concrete examples, but there must be a reason behind it. There must be an intention.

Dario:

Yes, I get the concept. Indeed an example is complex so, on the spot. It depends on the particular story, the single film, the single screening. It is a beautiful answer that makes total sense, though.

Because it is true that there are often so many things, or so many technologies, only done because you had to find a problem to the solution; not the solution to the problem. Which by the way, at the moment, is virtual reality: a totally useless technology the way it is structured that they are trying to find a use for.

Technology should help, not destroy

Andrea:

Yes, let’s say I am a bit, um, technophobic on these points. I really like analog. I’m one who read books, who have an agenda. In this respect I am a little behind.

There are some really cool virtual reality applications. There have been exhibitions here that I have gone to where it has been used really well. As there have been other really low-level things.

Dario:

In which a television set was fine.

Andrea:

That’s right, let’s say. In my opinion, the technology is still a few steps behind the ideas.

Dario:

It is true, exactly that. It goes a long way perhaps in gaming, because still we come back to the argument that spectacle, the visual of images, of the location, and the involvement do so much. But it’s the only area currently that’s really going… With education they’re trying to do something, they’re inventing, but we have to wait to really have value added to the traditional method that’s always been here, and maybe can be modernized with little.

Andrea:

Correct.

secretcinema.org, to learn about Secret Cinema

Dario:

Okay Andrea, I don’t know if you want to add anything else.

Andrea:

No Dario, thank you very much. It was a really interesting talk. Go to the Secret Cinema website (www.secretcinema.org, nda), and go see what it is. If you haven’t been, absolutely try to see it, because that are truly unique experiences that will certainly bring you closer to whatever movie you go to see at Secret Cinema.

Dario:

Okay. I also support the invitation; I have already gone to see the site and from that I want to move on to see the real Secret Cinema. So, I will have to go to London as soon as possible to do that. It’s really beautiful. An innovation like few in the world, and as such it deserves to continue.

Also to you Andrea good luck with everything. Directly for your career, your future, and any other innovative experiment you get involved in.

Andrea:

It’s going to be a very, very interesting year, I think. There are many interesting insights, so definitely maybe we’ll have a chat later on my new adventures.

Dario:

Definitely. That’s all I’m waiting for. Thank you, Andrea. Bye!

L'articolo Secret Cinema: immersive, but social è apparso primo per Future of Cinema - by Dario Riccio.

What is a media, or “mediated story”

Cinema belongs to the “media” category. Also definable as “mediated story”, or “telling a story through something”. They can be a microphone and a speaker (radio), a film camera and a projector (cinema), a camera and a television (TV), a word processor and a printing press (newspapers). Any way of telling a story that is not “direct”, therefore different from traditional conversations, is defined as media.

Among these, there are the new experiential media to offer users potentially positive and beneficial results, increasing their involvement and above all empathy. The experienced experience makes them feel active and involved in the story, seeing, hearing, touching or experiencing situations never experienced before.

The characteristics of traditional cinema

Each media has advantages and disadvantages. Or, we could say, possibility and impossibility. Going into details, let’s quickly see the five peculiar characteristics of the media at the center of our attention: cinema.

• Static and linear narrative structure
• Single or dual mode
• Episodic
• Third person perspective
• Passive audience

Static and linear narrative structure

It’s at the basis of cinema: a one-way narrative structure with beginning, development, climax and end. Fixed and static, it obviously cannot be changed by the viewer. Sure, there are Flashbacks or Flashforwards, but they don’t fundamentally change this approach to storytelling.

There is usually a causal chain: each passage narrated in the script leads to a subsequent passage which is a consequence of the previous one.

And the movies, of course, don’t change over time. Except for very rare cases of errors or problems occurred after the release of the film (for example Kubrick cut the last minutes of The Shining after its release in theaters), these remain the same from the moment of publication, and forever.

A good analysis was made in 1992 by Prof. David Pinault in Story-Telling Techniques in the Arabian Nights.

Non-linear structure in traditional cinema

In postmodern cinema there is actually a concrete attempt to modify this linear structure. Often attempts are made to “weave” the story, leaving the viewer to use intuition and irrational instinct to understand the film, while logic and drama are dissolved and diluted.

The linear structure often develops into a symphony composed of several parts with a non-linear structure. We will understand better in the following sections how this will develop even more in experiential / interactive cinema, forming various personalized narratives (which is reminiscent of videogames).

Single or dual mode

By this we mean that cinema involves a maximum of one or two senses, sight and hearing. Clearly the current cinema is always dual, as in the “single mode” category we can only insert silent films.

Over the years, various attempts have been made to improve the involvement of the other senses. We think of great directors who manage to convey the sense of taste to the viewer, for example in films such as Eat Drink Man Woman, Ratatouille or Mid-August Lunch. Clearly, it is a trick to our brains. But ultimately, cinema itself is.

Episodic

According to Prof. Jason Mittell in his book Complex TV: The Poetics of Contemporary Television Storytelling, current cinema and media are generally episodic; that is, they tend to develop around an event or a series of interrelated events. This is even more true, of course, in the world of journalistic media.

A story is told based on facts and events, real or fictional. It is told from the point of view of the narrator, which makes it easier for the viewer to enjoy but at the same time empathically distances him from the story.

Third person perspective

This perspective is an important feature in our analysis. In fact, although possible, in traditional cinema first and second person perspectives have always been little used. In the history of communication in general, it was abundantly used only in the radio of the first half of the 20th century.

Heart of Darkness, first-person radio adaptation

As an example, consider Joseph Conrad‘s Heart of Darkness. It was adapted for radio in 1938 by director Orson Welles (famous for having made Americans believe they were under Martian attack through the radio show War of the Worlds, in the same ’38). The goal was to have the protagonist tell the story in first person directly. Interesting how Welles himself, at the time new to cinema, tried to persuade RKO Pictures to make the film version.

It had what it takes to become one of the greatest films of all time and, perhaps, raise public awareness on issues that instead, misinterpreted, led to the Second World War the following year. But it was the use of the first person, in addition to the political themes little loved by the majors, that probably pushed Hollywood to not consider its feasibility. It was a drastic break from the rules, and the world wasn’t ready for it yet.

In fact, Coppola tried to recover in 1979 with Apocalypse Now, only freely inspired by the novel “Heart of Darkness” as it is set in Vietnam and not in Africa. It is certainly too late to assist peace in Europe and in the world.

First person in the history of cinema

There are some sporadic cases of filmic use of first-person narration, especially in its early years. Sometimes little known cases, but which somehow tried to change the way of seeing things. First of all I think of Dr. Jekyll and Mr. Hyde of 1931, better known in Italy as Dr. Jekyll and Mr. Hyde, by Rouben Mamoulian.

Other cases were William Dieterle‘s 1934 film The Firebird. Or even Lady in the Lake shot in 1947 by Robert Montgomery, and Dark Passage by David Goodis of the same year.

Two other examples, decidedly more modern, we will see shortly in the section on experiential cinema, as they are more useful for a comparison with the cinema to come.

I have cited only examples of works in first person, because talking about works in the third would be impossible or useless… They are practically almost all the films in existence. And then because, with a view to creating the cinema of the more experiential future, I believe that these ideas must be well taken into consideration.

Then you will tell me what you think, I’m interested in being closer to the technical world than to film criticism.

Passive audience

As we said, the story in the current or past media is told in the name and on behalf of the narrator. Whether it’s the writer or the director, the journalist or the speaker, everyone shows you what they want. Have you ever wanted to look at something to the side, or behind the camera, but the direction didn’t show it to you?

It happens particularly often with sport on TV, and it is perhaps for this reason that television itself is among the first media to have “progressed” towards a direction decided by the viewer. In Italy, as Wenner Gatta mentions, when the “Spidercam” was inserted on the football fields, the viewer could choose with the remote control whether to see the meeting with the classic direction or through the spider camera only. And from there, above all Sky has continued to develop technology a lot by exploiting multiple transmission channels for the same event.

After a study of the cinema that was, and still is, we are finally going to analyze the characteristics of the new cinema, projected towards the future, to also understand how to overcome the problems of the last 127 years.

The characteristics of experiential cinema: the future

The five characteristics that we will be able to find, all or in part, in the cinema of the future, according to what technology currently makes available, can be:

• Immersivity
• Interactivity, non-linearity and sociability
• Multi-sensory presentation
• Algorithmic, customized in real time thanks to data
• First person perspective

Immersive Cinema

We already know some partially experiential media. We think of immersive virtual reality and augmented reality platforms. As we saw in the last article, these are fully inserted in the line of continuity between real and virtual world hypothesized by Paul Milgram.

Immersion means “enveloping the user in a real physical space using augmented or mixed reality on a portable or wearable device, also including haptic interfaces“.

I found one of the first practical examples in the paper “Situated Documentaries: Embedding Multimedia Presentations in the Real World“, by Tobias Höllerer, Steven Feiner and John Pavlik and taken from the International Symposium on Wearable Computers of 1999.

These “set documentaries” were entirely based on wearables, to integrate novels and documentaries in real-world locations. The system is very reminiscent of the current AR glasses, obviously with the technology available in 1999. It was a backpack with GPS tracker and 360 ° video camera (developed by Shree Nayar, of Columbia University), a type of handheld computer with graphics, audio and video, and augmented reality glasses capable of signaling in the real environment the points of interest. It was called MJW (Mobile Journalist Workstation).

The glasses could also reproduce rudimentary 360 ° videos superimposed on the real, and the gaze was the main aiming system. By staring at an object in the real world for at least half a second, it was selected producing related info and multimedia files. You could also travel in time by touching the desired year on the handheld display.

A great example of what augmented reality will be 20 years later. AR is in fact developing rapidly, becoming in common use since 2018 following the implementation in iPhones of the native ARKit api (which, by the way, I believe to date is the latest noteworthy innovation in the smartphone field).

The first point of our cinema will therefore be immersion. But certainly a different immersion from that seen so far. The basis will in fact be the story told, the cinema. We will not immerse ourselves in the world, we will not immerse ourselves alone or in virtual common environments. The immersion will be produced by the 360 ° screen, by the stereoscopy, and by the “hall”, or dome, with inside elements directly linked to the narrated story.

The writers will have to be really good, to develop plots that make the viewer feel “involved”, without being taken for granted. As is obvious, the first thing that jumps to my mind: to keep the protagonist of the story constantly seated, who must impersonate us spectators. It is at best a starting point that should not be underestimated, but above all the brain storming will be really interesting in the early days.

Interactive, non-linear and social

Word to the viewer; choice. These may be the keywords for the non-linear structure of experiential cinema; which is therefore more complex.

Although without particular rules, it maintains its own logic based on temporal sequence (or order), duration and frequency, which are the concept of division proposed by the French essayist Gérard Gennette in the field of literary fiction, then introduced in the field of film criticism by Andre Gaudreault and David Bordwell.

The order of events in the stories

In this section we open a general parenthesis on the stories; there are in fact characteristics in common with the future immersive cinema. Gennette clarified that the event could have occurred:

• before the narration (analyses or flashbacks)

• in the future (therefore only announced or expected, the prolixes).

• Again, events can be narrated in a different order from how they happened (anachronia), used to make the story more compelling.

• More rarely there can be a movement between one narrative level and another (metalepsis).

An example is the “author’s metalexy”, a sort of passage by the author from external to internal to the story, or conversely if a character becomes a narrator.

Another literary example is of the poet Virgil, who “kills” Dido in canto IV of the Aeneid, or Diderot who writes in Jacques the Fatalist: “Who could prevent me from marrying the Master and making him a beak?”. Both examples taken from Armando Mollica Bonivento‘s doctoral thesis at the Ca’Foscari University (in Italian), which I invite you to read for greater understanding.

The duration of events in the stories

So, after the order (event that happened before, future event, narration in order different from the real or passage between different narration levels), we have the duration. Also definable as the “rhythm”, the “speed” at which events are told.

We can mainly divide it into four types:

• ellipses (very accelerated rhythm), with frequent chronological jumps;
• synthesis (relatively fast pace), in which a story is summarized in its main points. They can be of variable length.
• scene: relatively slow, it is the classic narration almost in real time. An example are the dialogues;
• descriptive: no progress in history, we stop to describe a certain moment.

Clearly, these types can be combined. We can have, for example, a synthesis inserted within a dialogue.

The frequency of events in the stories

Frequency is nothing more than the relationship between how many times a certain event occurs in reality (even invented), and how many it is told. If, in practice, the same event is narrated several times (or the same statement of a character repeated).

I leave you the link to an interesting article on the issue, from which the following image is taken.

The cinema of the future has an intersubjective non-linear structure

After this long interlude, let’s try to understand why the story, or the screenplay, is decidedly more complex for the cinema of the future. In fact, all these elements must be inserted within an intersubjective non-linear structure. That is, the story can go back and forth in time, in an environment common to other viewers who may want to make different choices from ours.

Intersubjectivity is the biggest problem to be solved in writing new scripts. In fact, if interactivity is a concept already well known thanks to video games, interactivity in common between several people, with necessarily a screen that shows everyone the same images, presumes that a small democracy is created inside the room.

The intersubjective non-linear structure has a great advantage: it respects the viewer. It grants the right to choose one’s own story, to judge the morality of some scenes. The viewer becomes the center of the film as well as a part of it. The current cinema is definitely too one-sided, and if it has lived unchanged for so many years it is only for the simplicity (inherent in the characteristics) of using it as a means of political and commercial propaganda.

Thinking about the average use of cinema, which is also and above all a moment of relaxation and not very active entertainment, we must not however fall into the temptation to insert excessive “gamification”, transforming it into a video game. I mean, nowadays we go to the cinema to relax with friends or family, to spend time without thinking too much. And choosing implies thinking … This is why interactivity must be limited and not even mandatory, and it probably won’t be the top priority in creating the cinema of the future.

We will see in some future articles why interactive cinema has not been successful in, albeit few, past experiences. But it is related to this.

Multi-sensory presentation

Experiential media may seem like something new in recent years, but it’s not entirely true. For centuries, humanity has developed its peculiar characteristics, improving the technology available in small but constant steps.

Wearable devices to engage the senses

According to what was reported by the Prof. John V. Pavlik in the book “Journalism in the Age of Virtual Reality“, the first experience of” wearables “devices can be summarized in the Chinese invention of the ring / abacus, wearable measuring instrument dating back to the Qing dynasty of the seventeenth century.

Following in Europe, in 1780 the “pedometer” was developed, a step counter, to arrive in 1965 with the American attempt (unsuccessful) to create the first exoskeleton (Hardiman) to allow humans to lift up to 650 kg.

In recent years, developments have certainly been much faster, also thanks to the logarithmic scale of the technology that hardly stops once it has started. To understand, did you know that in 2004, just 18 years ago, the GoPro wearable camera came out … And that it even used 35mm film?

Cinema stimulates us physiologically and sensually

My body is not just an object among all objects, but an object sensitive to all others, which reverberates at all sounds, vibrates in all colors and gives words their primal meaning through the way it receives them.

Maurice Merleau-Ponty in Phenomenology of Perception

At the beginning of this paragraph I inserted the trailer for the film Piano Lessons, Jane Campion‘s 1993 masterpiece. I chose it as an excellent example of how current cinema tries, in more or less orthodox ways, to deceive the brain in order to involve senses not directly involved (touch in this case). I also invite you to review the last article which talked about Matthew Shifrin and his Legos for the blind.

This is the magic of cinema. Art in this sector has reached unimaginable heights, even if you try to think about how to go further. How to materially stimulate the other senses. Although already in the 1940s, the philosopher Siegfried Kracauer wrote:

The material elements that appear in the films directly stimulate the material layers of the human being: his nerves, his senses, his entire physiological substance.

Siegfried Kracauer

How to involve the five senses in the cinema

How then to involve the five senses, or at least more than two, in cinema? It will be necessary to proceed in stages, as technology allows it. So let’s consider the experiments done in the past, to later understand how WE can engage the five senses for our viewers.

Touch

Meanwhile, the touch: William Castle, in 1959 shot the horror The Tingler. He inserted a vibrating device called “Percepto!” Into the seats of some cinemas, which was synchronized with the action. Castle himself was an evil genius … Before the screening of Macabre in 1958, he had everyone deliver a $ 1,000 insurance policy in the event of death or fear during the film. Then during the 1959 film House on Haunted Hill, he let a phosphorescent skeleton enter above the stalls. Based on a pulley system called “Emergo“.

And finally there is “Illusion-O“, launched with the film 13 Ghosts: all the elements in the frame, with the exception of the ghosts, were subjected to a blue filter. The ghosts instead had a red filter, and were superimposed on the frame. The audience received cards with red and blue filters: looking through the blue filter, you couldn’t see the ghosts. Through the red filter, however, they could be seen.

Still on the touch, obviously the modern 4D cinemas that we all know have the entire vibrating chairs, as well as ropes that touch the legs, fans for the wind effect (hot or cold) and sprinkles of water for humidity.

Smell

Smell: a strategy was adopted by John Waters for the 1981 film Polyester. This is the “Olorama” system, basically cards with scratch numbers. Each number has a smell (rose, pizza etc …), which can be smelled at the required moment (a small number appeared on the screen).

Another 1960 attempt was Smell-O-Vision, used only in Scent of Mystery produced by Mike Todd Jr (son of the famous Mike Todd Senior, author of Around the World in 80 Days). Smell-O-Vision involved the introduction of up to 30 evocative smells into the stalls through tubes that led to individual seats in the room, with perfume bottles held on a rotating drum.

Today a company has taken up the concept in a modern industrial production, called Olorama. With whom it would be nice to collaborate; objectively, the system seems much more functional than smelling cards (then actually reused only in a couple of children’s films).

The criticisms he received are interesting. According to the Times, some viewers complained of delays between the smell and the scene, others found the scents mixed in an unpleasant way, Henny Youngman said he didn’t understand the film because he had a cold.

Other inconveniences to pay close attention to are nausea and headaches caused by too strong and persistent fragrances, possible discomfort, distractions and heavy air.

For the sake of completeness of information, I quote Walter Reade Jr‘s AromaRama. Key difference with Smell-O-Vision? Simply, the AromaRama used the air conditioning system for the diffusion of aromas. Cunning.

Eating with another is a way of saying: “I’m with you, I like you, let’s form a community together”.

Thomas C. Foster

Taste

We have seen touch and smell, taste is obviously lacking. Although progress has been made towards systems of “transmission” of taste (through objects to be licked), I do not think they are yet developed enough and, above all, we are not yet ready to welcome them. And I don’t know if we ever will …

However, during some screenings of Willy Wonka & the Chocolate Factory, Wonka chocolates were provided to the spectators. And from this I had the idea of directly providing the screened food, really to the spectators. A double advantage: for the identification in the film, and for the cinema economy and customers who would respectively sell and buy better food than popcorn and Coca-Cola.

I soon discovered how this idea is not exactly original: in 2012 in London, more precisely in Notting Hill, Edible Cinema took place. It was a collaboration between the Soho House team, organizer Polly Betton and experimental food designer Andrew Stellitano. Basically, each present had numbered bags and glasses containing food and drink, on the armchairs there was also a menu explaining the meals, and a woman appeared on the side of the screen during the film to indicate the time to eat or drink each number.

Algorithmic, customized in real time thanks to data

In a data-centric society, cinema cannot stand by and watch. Of course, in respect of privacy and possibly without using this data for not very noble purposes.

A great use could come from the geolocation inside the dome, to eventually send different signals to the different spectators. But, above all, it will be possible to take into account the direction of the gaze of the latter to understand what is more interesting, and to have a more passive “input device”, and therefore less tiring, due to the interactivity we have just talked about.

Other anonymized data, such as physiological responses moment by moment, may be useful for the development of subsequent films and evaluate the reactions of the audience. Which, in a subsequent development of experiential cinema, can also be exploited within the same story (for example, interactively manage volumes to increase or decrease human reactions).

Women’s Aid, a prime example of algorithmic interactive advertising

At World Women’s Day in 2015, an interactive blow-up depicting the face of a woman victim of violence was installed in the Canary Wharf business center in London. A face detection camera was used to update a counter and change the image whenever a passerby paid attention to the advertisement. It was a typical example, albeit in the advanced advertising sector, of using data to modify the result obtained.

Artificial intelligence for storytelling

Already today many stories are created with artificial intelligence, as for example the American Associated Press does. The Times also created an algorithmic robot, the QuakeBot, which automatically acquires data from the U.S. Geological Survey, the US organization of earthquake analysis, to automatically write the complete article with magnitude, epicenter and time. The human editor only has to verify its correctness and publish.

These robots, these artificial intelligences, are led to the development of increasingly engaging, interactive and multisensory stories. What will be fundamental for the cinema of the future, to assist human screenwriters in the writing of increasingly complex and autonomously unmanageable stories.

Furthermore, they will be able to exploit the data present in the world, and in the cinemas themselves. To create “tailor-made” stories for the target audience.

Artificial intelligence itself is, and will increasingly be, used in technical video production. Google, for example, has the Jump compiler (well described in their own paper), which deals with the union, stitching, of 16 high quality video streams to obtain a complete 360 ° VR video. The main result obtained was the elimination of much of the latency, to therefore increase the sense of reality of the image shown.

Ultimately, AI will certainly be the center of attention. And, to avoid controversy, it must also be used with lead gloves, focusing on privacy and the importance of the human being.

First person perspective

We already know some partially experiential media. We think of immersive virtual reality and augmented reality platforms. As we saw in the last article, these are fully inserted in the line of continuity between real and virtual world hypothesized by Paul Milgram.

These are clearly in the first person, as the real protagonist of the story is ourselves. Experience is given by contact (although still virtual), and by direct observation of objects and events in the ways we like best.

Experiential cinema will often have to be in the first person. This is unlike most of the present and past films, which instead aim to tell us stories with eyes outside the story. But ours will be a different first person: we will be the spectators, the main character. Recently, in 2016, the film Hardcore, directed by the Russian musician Ilya Najšuller, had a good success. Evidently producing cinema from a cultural background outside of it helps to take risks and innovate it.

The whole Hardcore is shot in first person. I believe the success is well deserved, and it is a good point of reference for the cinema to come. A way of narrating video games, which invites us into someone else’s life, looking at the world through his eyes. Well, I think the only difference is that it will have to be our life, inserted in the new cinema. I know, it’s scary, but the writers shouldn’t tell us that either …

I also want to mention another film, not entirely in first person (the protagonist can be seen in various scenes, removing the effect of total identification) but which has a decidedly more constructed and engaging plot than Hardcore: Enter the Void, from 2009, directed by Argentine director Gaspar Noé.

Two films that I recommend you see, first because it will be a special and different experience. Second, to get a taste (albeit reduced) of what cinema will be like in a few years.

To identify everyone in the history of experiential cinema

How can you build a film for many, which can realistically represent the life of each of them? Once again, the writers will have to do a great job of multiple introspection. By creating engaging but generalist stories, inserting characters who will also be new to the protagonist in the story. No known relatives, perhaps a distant cousin we didn’t know we had. Nothing more.

In 2015, on the occasion of the fourth international conference on Design, User Experience and Usability in Los Angeles, Aaron Marcus collected in a very useful book a series of information on the current relationship between computer science, the virtual world and human beings. Based on the assumption that each of us has our own culture and knowledge, the design (and therefore who creates it) must leave us the freedom to experiment, to have doubts and thoughts, it must build a world accepted by all the personal cultures of the spectators and, at the same time, propose his own idea of the universe.

We want and need to reflect, to understand. And this absurdly leads to the hyperrealism, that is, to a construction of the virtual world that is very faithful to the real world, in which to experience and be present in the first person.

The importance of sociality in the cinema of the future

We have seen, speaking of the first-person perspective, that technology now allows us to decide a place, at a given historical moment, and live virtually in it. But one thing will be important: we will have to live it with others.

Sociality is a theme that should not be underestimated: there is no life alone. To reconstruct the real in the virtual, other human beings (and living beings in general) must therefore necessarily be present. And a concrete interaction with them.

Basically, reality is an inherent concept in our mind. Many things are real to you and me, while others are only real to one of us. The “mediated” reality has an inherent complexity in this, mainly because it must be able to reproduce a synthesis of our personal realities.

Let me explain better with an example: Aurora has a habit of calling her mother in case of problems. In her reality, her mother is always present. He gives her advice, embraces her, offers her constant support. Marco, on the other hand, has a bad relationship with his mother. She has always had problems, she has tried for years to help her but to no avail.

Aurora’s reality requires a loving and very present mother, on the contrary that of Marco rejects her. The writers of the new cinema will therefore have to be able to fall into hyperrealism, without causing moral incidents with any of the spectators. How is this resolved? Important characters may, or perhaps should, be ambiguous. An ambiguous character that allows everyone to “accept” it by mixing their own inner reality with that reproduced and therefore “mediated”. In short, see them as you wish.

Ultimately, the “static” part of the real world, such as trees and houses, is always there. And easily reproducible. On the other hand, the human part, the social part, represents a definitely more complex choice, even if not insuperable.

L'articolo What is experiential cinema è apparso primo per Future of Cinema - by Dario Riccio.

Current cinema, a modern derivation of the ancient analog, has five characteristics: static and linear structure, dual mode, episodic, mainly in third person and not interactive. We will see later what each of these voices means, while I continue to think about what the cinema of the future will be, about immersive cinema, and how to make a good prototype.

I started this article on a Swiss train, and finished it at Barcelona’s charming Espai Joliu. I love trains, being able to work or relax while moving between two locations. Among my corporate benefits is the general season ticket for all means of transport in Switzerland: my luck. A few days off from work, I take the opportunity to reach Domodossola (the first Italian town across the border), or wander the length and breadth of the country of the Helvetians, which reserves surprises and magical corners almost everywhere.

How important is immersion in the cinema of the future

I am part of the line of thought, now increasingly in vogue, which imagines a single evolution for cinema: immersive cinema. According to purists, or “traditionalists”, cinema must be left as it is. But we have already seen that, after all, they are not necessarily right.

As you may have noticed, I am not at all cadenced in publishing articles on my blog. I don’t like online editorial plans, I prefer quality over quantity, avoiding in-depth articles only to come out “Friday at 7”. This article took me days of research, maybe more, and clearly blogging is not my primary activity, but it has resulted in a roundup of ideas and characters that I hope will be a definitive answer to the question: how will the cinema of the future be?

Few have seriously tried to innovate it, and in this article we will get to know some of them. Certainly not for lack of a market. Rather for lack of resources, lately more and more projected to the lucrative, and more manageable, virtual world.

This article follows from: Innovating modern cinema

Advantages and disadvantages of modern cinema

I reflected on the advantages of cinema as it is conceived today:

• It can be used on the move, even here on the train.
• Other actions can be taken, delegating the mere role of companion to the audiovisual.
• It can be used in company.

I ask you for other advantages, I really want you to comment. Turning to the disadvantages:

• The methods of use have been updated little for decades.
• It is not able to “fill” all the senses of the viewer.

So let’s create a system, using already existing technology, able to minimize these disadvantages.

Analysis of the advantages and disadvantages of modern cinema

1) You can watch cinema on the go

Clearly, “old style” cinema by definition is destined for theaters. However, in order to recover costs, a production must be usable by as many people as possible. A good substitute for the small screen are virtual reality viewers, which allow you to view stereoscopic 360 ° videos.

2) You can do other things while watching a movie

Here virtual reality, at present, shows all its limits due to the complete isolation it creates. Bypassing this limit is perhaps possible with augmented reality, or with a serious upgrade of existing virtual reality technology.

3) You can watch a movie in company

By designing an update of the cinema system intended as a “cinema”, for decades there has been a technology used above all in scientific dissemination: the fulldome, ie projections in a dome. However, the same system is not new to film experimentation, even by great masters of the past.

4) The methods of use have been updated little for decades

The first disadvantage of cinema is rather an observation. If, on the one hand, great directors or companies have brought immense innovations in shooting methods (think ofAvatar di Cameron con una Motion Capture all’ennesima potenza, o alla Industrial Light & Magic che insieme ad Epic ha dato il là ad una seria Virtual Production), poco si è fatto per le sale.

The transition from film to digital, without wanting to diminish it, is comparable to the change of our home TV. Best quality, lower distribution costs, maximum stereoscopy but otherwise everything as before.

5) It is not able to “fill” all the senses of the viewer.

A century ago, a great innovator, today’s cinema fails to offer that “something more” compared to other more modern tools.

When looking at an iPad or modern TV, the field of view covered by the image is approximately 25°. The cinema screen, on average, covers a field of view of 50° on the 360 of the sphere (it depends a lot on the position in the room). It is necessary to increase it, if you really want to return to offer that “wow” effect, due to the immersion, which is worth the trip and the ticket.

The acoustics are already very good; Dolby Digital systems allow excellent immersion. What is missing is still the feel, the smell and, why not, also the taste.

Why virtual reality is not popular

I’ve been trying to figure out why you never want to put a helmet on, and a recent University of Glasgow paper / survey by Laura Bajorunaite, Stephen Brewster and Julie R. Williamsoncertainly helped.

The paper deals specifically with the use of VR viewers in public transport, which in itself is a place of frequent cinematic use. But the discourse is expandable, with the necessary adaptations, a little more generally.

At the base there are clearly reasons of personal safety, but also of comfort. Social acceptance is also still a long way off, making VR users “stupid” in the eyes of other passengers. Progress is still being made to make one’s virtual world more integrated with reality.

For example, occasionally throwing an eye to the real world with the cameras installed on the viewer, or having indications on the positioning of other human beings in real space, can reassure us. A complete Virtual Reality must be avoided in favor of Augmented Virtuality; as already defined in 1999 by Prof. Paul Milgram in its line of continuity from real world to virtual world.

I invite you to read the paper to learn more, but the concept we must take into account is that total immersion is a problem.

Immersive cinema

We have already covered the history of fulldome cinema in detail (article available here), but To the Moon and Beyond, is considered by many to be the first immersive film in history. The first to cover a 360 ° field of view, filmed with the Cinerama technique using a single fish-eye lens, on 70mm film at 18 fps, and projected onto the roof of a large dome about 30 meters high.

If its use was still somehow “didactic”, we moved away from the astronomical canons to bring instead a greater involvement towards the issues of transport. The film also influenced Stanley Kubrick, hiring its creators as VFX consultants for the legendary 2001: A Space Odyssey. Lester Novros and our next protagonist: Douglas Trumbull.

Douglas Trumbull, a mission: to save cinema

Douglas Trumbull was a man with a great idea:movies can be more realistic and more immersive than they already are.

I would have liked to meet him in person, he was a myth for me. I tried to write to him a couple of times, offering to visit him at his Trumbull Studios in the countryside of Massachusetts, but unfortunately I got no reply. Perhaps the addresses I had were no longer current, perhaps he was too busy as well as old. I don’t know, the fact is that he is no longer with us except through his studies that still have the time to change the history of cinema.

He was one of the greatest experts in the world of special effects, first analog and then digital. Creator of amazing graphic masterpieces in the top of last century Hollywood production (2001: A Space Odyssey, Close Encounters of the Third Kind, Star Trek …). After starting to smell the decay of traditional cinemas, he devoted his entire life to finding a solution. Which he too found in immersive cinema, as a true pioneer.

Perhaps due to his analogue habit of always finding physical and real ideas rather than creating a new app or a new system to monetize users’ time while they are alone or at home, Douglas was a multinational craftsman. He certainly had art in his DNA: his father, Donald (Pappy Trumbull to friends) had been involved with VFX since the 1930s. Not officially accredited, he worked on The Wizard of Oz from 1939.

It was during the filming of 2001: A Space Odyssey that, only 25, Doug created the legendary psychedelic color stream “Slit Scan” based on the technique developed shortly before by John Whitney for Vertigo.

Immersivity is the future of cinema

I report part of an interview made by Filmmakermagazine.com to Trumbull himself.

If you look at the industry, it is moving more and more towards immersive experiences. Take virtual reality, for example; the problem is that people don’t want to wear the headset. Immersiveness is what TV cannot offer; when 3D took off, everyone started making 3D TVs. But now nobody makes them anymore.

I think the problem was miniaturization. When you look at a 3D image on the big screen, you have a 6 foot (1.82 meters) figure standing in front of you, while when you put it on a TV screen you have like a 6 inch (15 cm) figure. When I took the projects we were working on and played them on the TV screen to see what they looked like, the effect was completely lost.

Douglas Trumbull

Magi Pod: innovating the cinema

Douglas Trumbull in his later years developed with Trumbull Studios a system that united many of the key points for the renewal of cinemas. A redesign of both shooting and projection systems. At its core, 120fps 4K projection solves a problem Kubrick already denounced over 50 years ago: 24fps blur is obnoxious and unrealistic. For this already 2001: A Space Odyssey was shot at a high framerate, although almost no room was equipped for reproduction.

Trumbull wanted to bring back the magic he himself felt in visiting the curved screens of the Cinerama in childhood. But technically there were many problems to be solved.

Troubleshoot 3D projection

To begin with, stereoscopic projection requires halving the framerate (or doubling the projectors, which would be uneconomical). This means 60 fps projection per eye.

Then there is another, more concrete problem: in 3D films both images (right and left) are recorded simultaneously. But they are then projected one after the other, alternately (right, left, right, left eye etc …). There is therefore a small difference that is not visually perceptible, but not negligible from our unconscious. Solution? Record the two images with a distance of 180 degrees / shutter, then alternately, as they will then be projected. This solves a second problem, which concerns the “cinematic look”.

The high framerate is not in fact used in cinema because it makes the product too similar to video and TV. Trumbull then concluded to increase the flicker effect originally given by the cinematic 24 fps thanks to the recording of the left and right frames of the stereoscopy with 180 degrees / shutter distance. In addition to reducing the luminance of the screen (in the Magi equal to 14 FtL, less than 50 lux), but not reducing the gain of the screen itself.

The entire Magi patent aims to increase audience involvement: it translates into projection in the Magi Pod.

A small cinema, 70 seats ll in the best viewing area; toroidal silver screen (curved both horizontally and vertically) with 3x gain to increase contrast and therefore immersion; 32 speakers for enhanced surround effect; rotary subwoofers capable of reaching 1 Hertz, creating a “4D” effect with the absurd vibrations.

Trumbull’s years of experience leading the technical departments of the biggest blockbusters, in a single, magical, immersive cinema. If knowledge must always go forward, it cannot be ignored in the design of the cinema of the future.

Cross reflections in fulldome cinema

By designing a fulldome cinema, the differences with the Magi Pod are many and fundamental. The Magi does not in fact fill a dome, but only a part of it. And you cannot simply “enlarge” the field of view to compose 360 degrees. There is a first big problem: cross reflexes.

Anyone who has worked with high gain projection screens will have realized that a silver, 3X amplified screen is a kind of mirror. Reflecting both the light coming from the outside and the same image projected from one side to the other in the case of a convex screen. A problem that also experienced Steven Spielberg’s 1991 production, Back to the Future: The Ride; fulldome re-proposition of the original Back to the Future.

By limiting the radius, and keeping the spectators within an optimal viewing angle (therefore in central positions) the problem is reduced to almost zero. But a complete sphere must necessarily be a white screen with a reflection ratio of less than 1, or the solution is still to be studied. And in this case we will certainly have to do practical experiments in the near future.

Immersion in the cinema of the future

In fact, almost every previous study we talk about in this article is in some way linked to the digital world. But, as you may know, I am a staunch supporter of sociability and the return to real life.

The room will have to resemble a space and time machine, where you will enter a totally new dimension for the duration of the screening, and from which you will come out with a complete experience. Above all, in company.

The world is heading towards the conquest of space, which will initially be within the possibilities of a select few. Others will want to try new experiences, and they will have to do so while necessarily being on the ground.

Five senses for immersive cinema

The viewer will have to be the protagonist, so the narrative will also have to change. We remember well: without disturbing the intuition, the sixth, we still have five other senses.

Today’s dual cinema manages to cover only two of them: sight and hearing. And, especially for sight, we can do a lot to improve the situation. Reading the number of Wired Italia of December 2021 (magazine that is always full of interesting ideas) I found this paragraph in an article by Henry Jenkins (lecturer at University of Southern California, director of the comparative media studies program at MIT in Boston for more than 10 years; not the last of the arrivals):

I am convinced that the metaverse will not be the future both because devices such as Oculus viewers are terrible (especially for those who, like me, wear glasses), and because it is not the visual experience that guarantees the highest level of immersion, but that audio and haptics, as evidenced by decades of studies and pioneers such as Metthew Shifrin and Nonny de la Peña.

Henry Jenkins

Jenkins’s idea, similar to mine in many ways, intrigued me to get to know musician Matthew Shifrin.

Matthew Shifrin: Legos and the cinema of the future

Matthew Shifrin is a young musician, blind from birth. Since he was a child, a Lego enthusiast, these allow him to experiment, touch, “see” without eyes. Thanks to touch, and therefore also to Legos, he can understand how the world works; as did. According to his eloquent example, he will never know what the Statue of Liberty is like without climbing it. And climbing the Statue of Liberty can be problematic… Therefore, a small Lego statue perfectly fulfills its function.

It is his thirteenth year of life. At the door rings Lilya, family friend. She approaches Matthew with light steps, looks at him and says cheerfully: “I have something for you!”. Matthew, confused, smiles at her as the woman places a package in her hands. The child opens it, touching a sheet entirely written in braille: these are the instructions for building Lego structures by himself.. His eyes shine, he is so happy. And he wants to get to work right away.

Lego for the blind

The story leads him to found, years later, “Lego for the blind“. An association, with its website, where you can download instructions like these for about thirty Lego products. And convinces the same company to include them in new products. He also created a system to help the blind to climb the rocks, really brilliant.

Definitely an interesting character; but you may be wondering why we talk about him in an article on the cinema of the future. I understand you, I was so taken by his story that for a moment I did it too. Then I remembered where I wanted to go.

In his TEDx Talk at Suffolk University, starting at minute six, he explains it well and in a very funny way.

Blind people in the cinema of the future

The point is: if the world manages to be immersive for a blind person, cinema can never be immersive in a similar or equal way to the real world if it is not for a blind person. The fact that the cinema of the future can be enjoyed in a better way also by people with disabilities, can only be an excellent and stimulating side effect.

While studying singing and accordion at the New England Conservatory in Boston, he continues to look for ways to make modern life more suitable for people like him.

In episode 3 of his Blind Guy Travels podcast, Matthew describes going to the movies with his friend Ben. The need to have Ben to describe the visual part of every single scene, and even thus not have a complete idea of the whole, leads him to create with his university a tactile device for the cinema.

With the support of the New England Conservatory’s Entrepreneurial Musicianship Department, Matt and his business partner have developed a programmable vibrating vest integrated with technology that mimics the sensations of overturning, flying and falling. This allows a new audience to effectively follow the Marvel Daredevil saga, a blind lawyer who becomes a superhero at night.

A few days ago I wrote an email to the conservatory asking for more details on this system, unfortunately without receiving an answer. I plan to comment on this article if I have any news in the future, it is interesting.

We will talk about experiential cinema

I have ready an analysis of the characteristics of current cinema, and of how they evolve in experiential cinema. How the latter is interactive, immersive, algorithmic and other important studies for the purpose of creating our cinema of the future.

But I decided to end the article here, it was getting too long and dispersed. I have already copied the text into a new WordPress post, which I will complete as soon as possible and publish.

I renew the invitation to comment, to share doubts and criticisms, and above all to participate in this innovation if you like it. You can also write to me, my address is dario.riccio@gmail.com. See you soon!

L'articolo Immersive cinema is the future è apparso primo per Future of Cinema - by Dario Riccio.

How marketing was born

The progressive decrease of media censorship in the Western world, which began in 1766 in Sweden and has spread to the present day (at least, until a few years ago…) has favored the birth of a large number of publishers. The number of publications increased considerably, newspapers were increasingly present and communication became a real branch of the economy.

Until the 90s, the situation was still manageable and predictable by a marketing employee. There were some local and some national publications. First on paper, then on radio and finally on TV. With a brief exploit of newsreels, when the radio frequency image was still a future dream. The medium changed, but the language remained similar all in all.

Advertising agencies were born in the period of the industrial revolution, when for the first time the supply of goods exceeded the demand. It was therefore necessary to find buyers for this sea of products, to convince them that a product was better than its competitors. Initially, in the 19th century, it was still quite simple, even though there wasn’t a past to draw on. It was then tried with some paragraphs on printed paper, some posters in the corners of the cities and the product was brought to the knowledge of most.

The first advertiser in history, founder of the French weekly La Gazette in 1631, was Théophraste Renaudot. A visionary close to the poor, who in addition to having founded what we can consider the first modern magazine in history, also created a sort of temporary agency to reduce the poverty of the most needy.

Finally, returning to advertising, the higher the budget, the more local newspapers where one could advertise. Up to being able to afford nationals, or even better to expand to other nations. Basic.

But what about cinema? How were films advertised in the last century?

Marketing for cinema in the twentieth century

Film advertisers of the early years (first from French, then New York and finally Hollywood), began by copying theatrical advertising, only to understand that they had to conform to traditional industry communication.

In fact, initially the idea was to sell “an experience”, not a product. The cinema was sold more as one would sponsor a nice trip, rather than as a nice car.

Not least, early film production was not very regular, with no precise predictions as to whether, and when, the film would be distributed. Without having big ideas about the future proceeds and the best duration of the projections to maximize profit.

Today it may seem absurd, but we must consider that at the time there were not many data to draw on. It was a new world, made of experimentation.

Cinema from product to lifestyle

Having to sell a product by definition unnecessary discouraged even the best advertisers. No one had the physiological need to see this or that film in particular, nor a film in general. After numerous experiments, and gradually, it was realized that an experience had to be sold; a community had to be created. This need to see the movie, had to be somehow linked to the primary needs of man.

It was necessary to create in the public urgency, FOMO, to buy a product that could only be appreciated later.

And to do this, it was important to leave a strong and proper message to society.

Changing the lifestyle, creating movements, ideologies. Having a very strong impact on the society of debt and consumerism, deliberately wanted by the United States government after the Second World War to revive the fortunes of a severely tested economy.

That consumerism that managed to improve the lifestyle of Americans first, and then of Europeans, until about the 80s of the last century when the “American dream” ended. Cinema supported this dream, and the cinema communications workers wanted to break into the hearts of the population.

Collectibles and community marketing maturity

So not just living rooms, not just TV rights, not just VHS and DVD. The culture of “gadgets” developed, collecting, image rights on any object with the shape or name of your film or of your protagonist.

After these first tortuous decades, cinema marketing finally achieved the desired results. Cinema created fashions, and then followed them. Starting in the 1950s, there was a great crisis: the birth of TV caused a strong loss of turnover for cinemas. But Studios were getting closer to their audience, starting to design films that followed the current trends. Motions of rebellion, fun, fashion, rock ‘n roll… All to be able to conquer the public, if not in the hall, at least on home TV.

TV itself then went from being a disadvantage to a great advantage. In fact, the use of advertising in motion and trailers turned out to be ideal for cinema; who then, is none other than the precursor of that same daughter and traitor television.

Cinema Marketing – Recommended Reading

Books in English

For more informations, there are various studies about marketing in cinema. Some more focused on the practical part, a kind of how-to, like Movie Marketing: Opening the picture and giving it legs by Tiiu Lukk (Los Angeles: Silman-James Press, 1997), Open Wide: How Hollywood Box Office Became A National Obsession: Inside the Blockbuster Movie Factory by Dade Hayes and Jonathan Bing (New York: Miramax books/Hyperion, 2004) or Marketing to Moviegoers: A Handbook of Strategies Used by Major Studios and Independents di Robert Marich (Boston: Focal Press, 2005).

Other works are more theoretical, ideological. Like Global Hollywood 2, by Toby Miller (London: British Film Institute, 2019), which studies the influence of cinema, especially American, on global society.

Studies available for free in PDF

An important 2013 study by Sheldon Hall, freely downloadable from this link (Sheffield Hallam University Research Archive website), entitled “Jerry Pickman: The Picture Worked. Reminiscences of a Hollywood publicist”, cites classics such as The Greatest Show on Earth (Cecil B. DeMille, 1952), Shane (George Stevens, 1953) or Psycho (Alfred Hitchcock, 1960) and famous people like Adolph Zukor or Alfred Hitchcock, while highlighting the human aspect of marketing.

It reminds us that film advertising has its uncertainties, but it is also a human endeavor involving individuals such as Jerry Pickman, the great advertiser who died in 2010 and has dedicated over 50 years to marketing in film. Whose dedication can turn even an ordinary film into an extraordinary success. Last but not least, this seemingly unique vision of the film business shows us how film marketing developed on an industrial scale in the early 1980s, when the concept appeared that “now you sell movies like you sell hot dogs, as everyone does”.

A final reading tip comes from Stanley Kubrick. It is interesting to see marketing through the eyes of a director, to understand how art and business can be reconciled. “Stanley Kubrick and the Marketing of Dr. Strangelove” (1964, PDF downloadable here). Sometimes these interests converge, often not. In this text Peter Kramer, drawing on the archives of the London University of the Arts, describes how Kubrick was directly involved in the American marketing of Doctor Strangelove, from the earliest stages. To the point of committing to neutralize any competitors.

During the production and the first promotional campaign of the film, the famous director was very skilled in self-promotion; although the marketing materials used for the marketing campaign following the release of the film had focused on common topics such as celebrities, comedy and sex.

The beginnings of Social Media Marketing for cinema

In 2013, the Toronto Silent Film Festival was among the first to use Instagram to promote itself. The method was studied by the Canadian marketing agency Cossette, and it was very different from the usual ones: they uploaded frames of silent films on the American social network, which by scrolling the page went to compose the film in the eyes of the user. Original method, although it did not acquire a large following due to the premature state of the social network at the time.

Out of curiosity, here are the accounts: tsff_1, tsff_2 and tsff_3.

Also in those early years of social media, people were looking for ways to bring traditional promotion to them. Think of trailers, which however sometimes had to be drastically reduced given the duration limits of the videos. With today’s eyes, once again ineffective systems even if used. As if to say, “at least I’m on social media”.

The film becomes functional to marketing

Similarly, more functional campaigns began to be designed. Staying on the subject of Instagram, the film about Steve Jobs of 2013 saw his official account, with pieces of the movie, frames, short videos and photos with meaningful phrases. In the same years, new aspect ratios for films began to be experimented, at least indie ones, without renouncing much-felt criticism from cinemascope purists.

All to get closer to the public, who changed habits: from TVs and PCs, the most used screens became those of smartphones. An attempt was made to use the square format, as did Xavier Dolan in the movie Mommy. To then quickly switch to the portrait format, able to fill small portable screens in the best possible way and reduce the distractions of the spectators.

In recent years there has also been a shift to real blockbusters in vertical format. Long-lasting vertical videos, however, can be less usable, or cause loss of concentration, given the lack of filling of our peripheral vision. Equally, the Russian director Timur Bekmambetov in 2021 realized his V2. Escape from Hell with scenes in portrait format, using the screenlife technique (show images as if they were taken from a smartphone, even in selfie mode) [UPDATE: this is a typo, it was just an idea later abandoned to make the film vertically], and following the series distributed in 2019 exclusively on Snapchat Dead of Night.

Already an exclusive Snapchat film distribution, it shows the many attempts that directors, producers and companies were, and still are, making to approach the modern world.

The importance of human sociality

The era of influencers

What was missing in the early years of social media was, paradoxically, the social side. People want to know other people, not see advertisements. They want to communicate, feel empathy for someone, feel less alone. This led to the natural evolution of social media marketing: the beginning of the era of influencers, and of influencer marketing for cinema.

Today is the era of influencers. Which, among other things, are also becoming increasingly important in the world of culture. It is no longer a film critic, a trade magazine or an important media who sanction the success of a production; also due to the increasingly marked lack of that ancient journalist / reader trust, lost after decades of partiality (for almost always economic interests) of the publishing world. And that continues to destroy itself even today, as evidenced by the attitude of many users on issues such as the Covid-19 pandemic and the war in Ukraine.

Instead, it is people on social networks who are listened to with confidence; first and foremost the influencers, but also those who count less, “normofollowed” users who break even more forcefully in the hearts of friends and relatives.

In short, if Gianni says I have to see this film … It means that I have to see it.

The actors / influencers

So is influencer marketing suitable for the film industry? Of course, reviews in traditional media such as print or television and radio still play an important role in gaining publicity for films and television productions.

However, we have seen that the role of traditional media in cinematic communication has waned in favor of more genuine interpersonal communication (the good old word of mouth, technologically revisited).

The best-seen “advisors” are cinema enthusiasts, independent. They usually know their audience, first of all because they are similar to them and also because of the continuous interaction through chat, comments and quotes. The confidence given by having communicated directly, privately, with one’s “myth” is unparalleled (those of you who have been to dinner with a famous person can more or less understand the effect). And they often know how to offer original perspectives that no one else can give.

Now, let’s take it one step further. Let’s imagine for a moment that the influencer we are following, with whom we are actively speaking, is an actor in the same film. Or, let’s say, even just a technician. And who, like a friend, makes us follow the various stages of production, behind the scenes, even asks us for opinions on the story and, if we were experts, on the technique for the production of the film itself. How would that end? We would certainly get a discount or some advantage, given the “friendship” between us. Were it also the t-shirt, the hat, the keychain, even the possibility of being present on the set or at the premiere, or being part of the cast perhaps with an appearence. The possibilities are endless, as will the desire to go and see “our” film. The film in which a friend participates. The film we took part in ourselves. And what do you think, wouldn’t we tell everyone? Here, this is the heart of social media marketing for cinema. The marketing of trust, the return to word of mouth and closeness between people.

The importance of engagement among influencers

There is great collaboration in the influencer community. First of all they are colleagues, and among colleagues we understand each other. Then there is a mutual convenience to get to know each other, especially to be seen together. The fans of one become fans of the other, the possibilities for earning and growth become more numerous.

So, once you are full of influencers in your production, it becomes almost automatic to involve outsiders. Other people who also need to be emotionally involved. Then, invitations to the VIP area and front rows to previews, dinners with the crew. As for the fans, but the activities must obviously be more “personalized”. The influencer is used to being considered important, and it is therefore more complex to make him feel “better” than others.

It is important to have your actors interviewed by other influencers, similar to the old press. Also target cinema-savvy bloggers, podcasters and Youtubers. They too will be happy to be able to create interesting content, and at the same time make themselves known by the fans of their little stars.

Certainly personalities connected to the film industry are important, but we absolutely must not limit ourselvesto these. There are many people, many influencers, even small ones, who adapt for various reasons to that particular product we are making. And, as a result, their audience will be targeted.

Another advantage is that all of this is cheaper than advertising in the past. It requires more effort, mental but also physical, but with the choice of the right people upstream, the results can be unimaginable.

Only one ethical doubt remains: a good actor, but not very capable of socially involving an audience, may no longer have the possibilities he had in yesterday’s world.

Does influencer marketing for cinema have a future?

This is just a final parenthesis … Because here we are planning the future, and it is right to be critical. I’m not here to sell you social media, but to think together.

By now the Elites have understood very well the importance of social media. This could perhaps be bad, because those in charge always have an ancient idea in mind: the people are idiots. For heaven’s sake, I could almost say it’s true. We let ourselves be turned around like socks every day by the various propaganda (just think of this period, I write during the invasion / special operation in Ukraine, the name changes with respect to the speaker).

What’s the problem, aka luck, in all of this? That among billions of people, every now and then someone drinks a glass too much and starts thinking. And an article comes out in some strange blog or semi-unknown social profile. What others, even just 10 people, read. And if something is patently absurd, and that article denounces it … These people share it. And then again, and again, in a logarithmic scale of shares that leads to filling the web in a few days.

I was saying, problem and luck. Luck, because there is no longer the popular myopia of the past. Eventually, many things become clear even though the politics of politically correct leads to not reacting. We stay quiet, but at least we know. Problem, because this scheme is artificially reproducible. And those involved in political propaganda know this well … We therefore arrive at the paradox that it is no longer the journalists funded by the System who tell us what the System wants us to know, but our own friends. Reporting fake news, or rejecting real news but it is not good to disclose.

Will this be a problem for social media? I’m afraid yes, in the future. It will take time, but like print and TV yesterday, friends online will lose our trust one day. Socially, it is worrying on several fronts. But from a business point of view, let’s start thinking about what comes after the era of influencers.

And we keep the quality. Which, beyond all marketing, is the key to success.

L'articolo Influencer marketing for cinema è apparso primo per Future of Cinema - by Dario Riccio.

A more democratic cinema

Virtual Production (I also had to write the entry on Wikipedia Italy ), motion capture, internet, digital and above all low-cost training, allow you to no longer be blocked by heavy budgets. Even new directors, or technicians, can have their say with a sustainable model. In the past, low-budget often meant super-low-gain. Without the support of public funding (which especially in our latitudes bring forward only Chippuò‘s friends and relatives), it was impossible to think of producing a cinematographic work, even a medium-level documentary, without losing thousands of euros.

All this reflected on the costs to the public, dictated by the large lobbies that made it inconvenient for many poor families to enjoy the Seventh Art. A day at the cinema for a family still has a cost comparable to months of Netflix. The benefits of digital distribution are enormous, but this isn’t meant to be the only method.

Personal skills are more important than budget

A great one-man-band, today he can potentially create a movie on his own. And above all, a good quality film. The participation of influencers can even help with distribution (I’ll do an article on this soon, now that I think about it). It was unimaginable until a few years ago, right?

Of course, it has always been possible to make no-budget cinemas. Thousands, perhaps millions, of young directors have done so in the past. But we were talking about very different results: without rooms worth tens or hundreds of thousands of dollars (Red et al.), Without international studies, without huge teams of specialized technicians (which were rare and expensive, given the inherent difficulties in training), he was at the maximum possible to create films that are not on the market and of a quality tending more to the low level than to the medium. Technical quality, but also limitations in storytelling caused by the technical impossibility of reproducing many ingenious ideas.

Today, this is no longer the case. A well-armored computer, with a top graphics card and a real-time render engine (among which the most popular in the cinema field, Unreal Engine, is distributed free of charge …) allows you to obtain 3D graphics that have little to envy to the big . And they can make small producers grow a lot in the cinema of the future.

Technologies for the cinema of the future

Let’s move on to analyze some of the technologies that could help us in the new cinematic use. Initially I expected to insert more than one already in this article, but given the length of time I preferred to limit myself to talking here about the fulldome video, promising myself an article in the coming days that will continue the discussion.

Fulldome video

Fulldome video projection is a technology that I have supported for years. It is the visualization of a video projection on the roof of a dome, to have a hemispherical overview. One of the best ways to feel “inside” a story, without using (and introverting) virtual reality viewers.

Brief history of planetariums

The technology is far from new. It is in fact used for scientific dissemination in planetariums all over the world. The oldest dome-shaped construction known today dates back to 500 BC, built by the Etruscans. The history of art, and the visit to churches and ancient structures in every corner of the world, teaches us how this form has often been used in various eras to represent the sky, Paradise and the firmament.

To know the first “working” planetarium (I mean, in motion), we had to wait until 1229 when Frederick II of Swabia, Emperor of the Holy Roman Empire, he brought back to Italy a tent with a domed roof on which a mechanical planetarium was hung: a clockwork mechanism, with a globe in the center (representing the Sun), from which some arms that support the planets then known depart. To learn more about the history of mechanical planetariums, this book from 1965 is interesting (the reading is free and open).

The main disadvantage, or difference, of the first planetariums was the depiction of the sky in reverse: the observer saw the solar system as if it were outside it, and not on the Earth. This was still the case in 1774 when the astronomer Eise Eisinga built, by himself, what is today the oldest planetarium still in operation. You can visit it if you pass through the city of Franeker, in the Netherlands.

Between the nineteenth and twentieth centuries, the innovations were many and rapid. The birth of Carl Zeiss in 1846 drastically increased the quality of optical instruments (which before then were produced without preliminary design, let’s say “by trial and error”).

We then moved on to ever more precise mechanical planetariums, and the Atwood Globe in Chicago made history. A metal dome of about 5 meters built in 1913, with 692 holes to represent the stars, and a lamp to represent the Sun. For the first time, the concept of the planetarium was the modern one: the viewer saw the sky, as if it were on Earth under the real celestial vault.

The first modern planetariums

During the twentieth century, the evolution continued, the planetariums went from rare instruments available to kings and scientists, to places of scientific dissemination open to all.

The first projection planetarium was built by Zeiss, which in 1923 tested the first Carl Zeiss planetary projector in its 16-meter dome. Even today, obviously updated, they are used by some large planetariums as an alternative to digital systems. In the following years, planetariums opened in Munich, Rome, Moscow, Stockholm, Milan, Hamburg and the first non-European: the Chicago planetarium. This went on all over the world until 1983, the year of the digital turning point. The turning point, which led me to talk about planetariums in an article on the cinema of the future.

How the fulldome video was born

Here we are at a fundamental point: how the fulldome video was born.

1983, we said. We are in the States, in Richmond. More specifically, at the Virginia Science Museum. Here the world sees a digital video projector in a planetarium for the first time. It was different from the projectors we are used to today: the Digistar I, manufactured by Evans & Sutherland, used calligraphic projection. I admit to being ignorant on the subject, never in my life have I had anything to do with this technology. And the short Wikipedia entry isn’t much helpful. Of course, it was a vector (therefore, by definition, unrealistic and poorly detailed) and monochromatic (created by a single laser beam) projection. Can anyone help us understand the principle of operation in the comments? It would be interesting, out of pure historical curiosity.

The fact is, then the 90s arrived with the first DLP and LCD projectors. In 1996, Japan’s GoTo Inc. gave birth to the Virtuarium, specifically designed for projection of stereoscopic scientific videos in a planetary dome. Two years later, Sky-Scan introduced SkyVision, the first fulldome digital animation.

The equipment was still prohibitively expensive, but we were close to what is now a digital fulldome planetarium. And that it could become the cinema of the future.

Cheap fulldome cinema: is it possible?

The problem with digital planetariums was the cost of the equipment. A high resolution and good contrast video projector in the 90s and 2000s had very high costs, tens or hundreds of thousands of euros. Even more so with a good fisheye lens able to project on a hemispherical surface without noticeable distortions. To solve the problem we must therefore go back a little, indeed a lot … On October 20, 1939, when New Yorker James S. Conant submitted to the patent office what would become three years later the patent US2299682A.

Fulldome projection with convex mirror

The patent dealt with a shooting and projection system (photographic, for the time) that we could define as “Newtonian”, from the name of the inventor of the reflection telescope. The technique was not used in planetariums until 2003, when Paul Bourke, an Australian university professor, retook it to adapt it to the then growing business of small digital planetariums.

The principle is very simple: the image is projected through a classic lens for flat screens (a normal video projector, so to speak) towards a convex mirror very similar to the surveillance mirrors that until a few years ago could be seen in the corners of every supermarket. This mirror then deflects the image towards the dome ceiling, allowing it to maintain the right proportions.

To maintain good details, it is important to avoid secondary reflections due to ghosting as much as possible. This is why we usually avoid cheap surveillance mirrors, preferring (better to say, necessitating) first-surface mirrors. They are mirrors with a reflective surface (silver, usually aluminum or silver), superimposed on a support. The opposite of what happens in normal mirrors (called second surface) which have the reflective material behind another transparent such as glass or acrylic (to protect it). For more information, the mirror manufacturer Abrisa has written an interesting article on the differences between the two types of mirrors.

The problem with these mirrors is the delicacy, and the risk of blackening of the same (which must therefore be treated periodically). For the purposes of a fulldome projection, silvering with aluminum, less expensive, would also be good, since the system used for Newtonian telescopes has scientific needs of a very different nature (to maximize the frequencies of the reflected light spectrum).

Doubts of the scientific community

The Planetarian magazine, edited by the International Planetarium Society, in the March 2001 issue published various comments about the probable use, in the future, of planetariums as a place of entertainment unrelated to scientific purposes. Interesting Ken Miller, when he mentions the owner of a small cinema who, after trying various technological innovations, claimed that the cause of a bad income is always a bad movie, a bad script and a bad acting. The message counts, not the medium. And in this I agree in part. I can argue from my experience that the same message by a better medium remains a better message.

Will the cinema of the future be fulldome?

In this article it is still too early to give a definitive answer. In the coming weeks we will analyze other details and other technologies, but I remain fairly convinced that the fulldome can be a great compromise in the 21st century. Technology has accustomed us to increasingly immersive tools, and investments in virtual reality and augmented reality in recent years are pushing in this direction.

I was very excited about virtual reality, but lately I’m starting to think that its future may be less rosy than expected. I await Apple‘s entry into the sector to get a clearer idea, for now the main sponsor is Meta Inc., a.k.a. Facebook, a company that despite common opinion has never stood out for its ability to predict future. A company that has grown thanks to a single good idea of its founder Zuckemberg and his ability to attract investments, but which has subsequently developed only thanks to the acquisition of good, almost alternative ideas that work well in the same sector.

Virtual reality will still have a good success for the purposes for which it was initially designed. It has strong potential in gaming and in cultural and educational contexts, and its media push may serve to increase interest in an innovation of storytelling in images to get closer to a “collective virtual reality”.

As I imagine the situation to date, it will be augmented reality that will take us the most time during the day. An analysis on the use of AR for interactivity in the cinema of the future will also be fundamental.

As usual, I would be very happy if we could create a debate. I therefore invite you to comment, criticize or give new ideas under this article. And thanking you, I give you an appointment in the next few days for a next trip.

L'articolo Fulldome video: the cinema of the future? è apparso primo per Future of Cinema - by Dario Riccio.

I come from an interactive exhibition on Deep Fake in the arts, hosted in the pavilions of the Polytechnic University of Lausanne (EPFL). This Swiss institution, which in 2019 developed the technology for the recognition of “human” deepfakes, is a world-class excellence in the field of science and technology. He wants to help us understand the future of technology in art.

The theme is very topical; digital allows you to create artistic fakes that are almost indistinguishable from the original. But it also allows the use of the works away from their places of origin, to circumvent political or security problems. And it is on this that the organizers have concentrated, to evaluate the ability of a copy to cause lasting emotions over time.

The tools of technology in art

In a modern exhibition, interactivity is everything. For years I have supported the “museum”, intended as large rooms / parking of objects, dead in most cases. Exceptions are some masterpieces that still today attract thousands if not millions of experts and onlookers for their fame, but this is not the norm. In Switzerland, over 75% of museums do not reach 5,000 visitors a year. But even in Italy, attendance has increased in recent years (excluding the pandemic period). This suggests that we are moving in the right direction to modernize the service offered.

The tools used to add technology to the world of art are mainly touch screens, holograms, mixed, augmented and virtual reality.

Multi-view, or multi-viewer 3D projection

Mea culpa, I was not aware of a beautiful technology: multi-view, or multi-viewer, 3D projection. Operation is simple on paper, albeit technically complex due to the huge amount of data to be processed. A video projector, specifically the Insight 4K HFR 360 (toy worth over $ 300,000), projects 360 frames per second in 4K resolution. This exceptional framerate, combined with active 3D glasses (obviously synchronized with that framerate), allows for the simultaneous reproduction of 3 different stereoscopic videos, giving 60 fps to each single eye.

I tried it and, apart from the inherent discomfort in wearing active glasses for a user of “normal” glasses like me, the vision is perfect.

To further improve the situation, 6DOF tracking (with six degrees of freedom, understands where we are looking and where we are in the room). System similar, if not the same, to those used for the HTC Vive VR headsets (or the old Oculus). In simple terms, the 3D video changes perspective as we move around the room, to achieve the effect of “seeing behind” the projected objects. A change of parallax, copied from virtual reality. And here too, a nice way to add technology to art.

A note on the protagonist: the digitization with laser and scanner techniques of the Abbey of Monte San Michele in Bamburg (Germany). Closed in 2012 to be restored following major structural damage, it can therefore be virtually visited.

Automatic 3D photogrammetry with CultArm3D

In an increasingly automated world, where humans are needed less and less (does this seem like a problem to you?), It is no longer impressive to see a robotic arm photographing a mask far and wide to recreate it in 3D. It’s theCultArm3D.

It is specifically a 3D reproduction of the Samurai mask from the Edo period. And this is the world’s first scanner capable of producing a perfect 3D copy using automatic 3D photogrammetry. Produced by the research of the Fraunhofer Institute for Computer Graphics, of Darmstadt (Germany), it is able to calculate the size of the object, to scan it (even in the case of reflective materials), to recreate it in real time in color in 3D and then to be used in virtual or augmented reality, or printed.

Realistic 3D printing is possible thanks to the development of adaptive color 3D printing. In this specific case, the technologies of the Israeli Stratasys are used. All with a declared goal: to show ancient works without the risk of ruining the originals.

As I see it, well but not very well… Poetry is missing in such an object, and I don’t know how many would substitute it for the true original. The fact remains that in cases other than printing, such a machine allows significant time savings for the digitization of world cultural heritage (and in Italy, there is a lot to do).

Artificial intelligence puts us on the alert

This is what is first imagined when thinking about the future of technology in art. Nice, and at the same time a bit terrifying, Trust AI invites us to sit in front of her, take off the mask so that she can see better and answer her questions.

Her face is a hologram, she informs us of our personal data (age, sex, hair color) and progressively pirates our face until it assumes our appearance. In her speech, after having “stolen” her personality, she is keen to alert us to the data we share online; how these can be used for less than noble purposes by thieves.

Does artificial intelligence cause dangers? Certainly yes, but no more than many other technological innovations. And art installations like this help the public to understand how it works, in order to better defend itself.

The work was produced by Bernd Lintermann and Florian Hertweck at the ZKM | Hertz-Lab, in response to the disturbing spread of deepfakes. I invite you to read the long article in which they describe the project; it’s definitely interesting.

Augmented reality to visit historical monuments

Beautiful, fun for children, but apart from that, visiting a cave reproduced in 3D, displayed in augmented reality with an iPad in hand, I don’t think is the best way to use technology in art. I find it useless, a pure exercise in style; I do not want the creators who have indeed been exceptional.

Mind you, we are talking about one of the Mogao caves, near the city of Dunhaung (China) dating back to the Tang dynasty. Therefore, something very ancient endangered by the excess of tourism. Digital representations are welcome, but this is 100% the field of virtual reality: a trespassing of AR does not produce the same results.

Holding the big tablet in your hand is uncomfortable, to begin with. And since it’s all reconstructed in 3D, why not take advantage of VR to feel like you’re really there, and not walk into a black cube with white textures drawn on the wall?

Augmented reality can really do a lot to improve the quality of a tourist visit, but in the real place. You can get help from an iPad to reproduce life scenes from the past, legends, descriptive videos of a particular point… But a mere virtual visit, let’s do it right.

Augmented reality that makes us think

As soon as you enter the exhibition, two works arouse curiosity. The “Reclining Pan” which we will talk about shortly, and an isolated white pedestal. This is the basis for “The Golden Calf” by Jeffrey Shaw. A calf that takes shape only by pointing the object with an iPad which, thanks to augmented reality, reproduces it.

The importance of The Golden Calf is very clear thinking about the year of production: 1994. Shaw was a true pioneer in the world of augmented reality, and obviously in its original form there was not an iPad but a screen equipped with motion tracking. magnetic Polhemus, one of the very few companies to offer this technology at the end of the last century.

The most interesting effect are the reflections on the calf. Four cameras placed on the sides of the pedestal reproduce our image on the subject to give it that extra touch of realism. And to make the work always different, to want to make the viewer himself a co-author. And the calf, elusive, is the object of our desires.

3D printing for the protection of the sculpture

The “Reclining Pan” is the modern copy of an ancient copy. The original work is attributed to the Florentine sculptor Francesco Da Sangallo, built around 1535. In turn, it was a copy of which, however, the original has been lost.

The contemporary artist Oliver Laric made a 3D scan of it (which you can download for free from Three D Scans), then printing it in many different materials. The result is a new, modern, and provocative work. It calls into question the veracity of the post-original in the modern era, rejecting the postulate of the singularity of art and its private property.

Laric, for his popular work, has made many works scanned in 3D available, free of charge, to anyone who wants to download them from the Three D Scans website.

New paths for the future of technology in art

I have not described here all the works, which were over 20. I wanted to talk about those that most of all make us reflect on the changes that technology is bringing to the world of art. In addition to these, it is also worth mentioning the blockchain which here was represented by the multimedia work “89 seconds to Alcázar“, by Eve Sussman. Videos whose rights have been divided into 2,304 “atoms” of 20 × 20 pixels, and sold via blockchain. And whose EPFL reproduction was full of holes, as some atom owners did not consent to free public viewing.

A bright future, in which the public domain will be increasingly important. But at the same time, a future in which artists will have more and more ways to experience their creations, to spread them to the whole world.

In which it will be easier to organize thematic art exhibitions, even in small realities, without the enormous costs of transporting works of great value.

In which spectators will be able to admire and study works of art from all over the world, and from all eras, without the need to travel thousands of kilometers. Which, for heaven’s sake, is beautiful. But sometimes, and for some, impossible.

L'articolo The future of technology in art è apparso primo per Future of Cinema - by Dario Riccio.

On a break from work in the Swiss mountains, under the gray sky of early February, I lose myself thinking about modern cinema. What he sees in empty halls (and no, Covid is only partially involved). The one that sees series work more than feature films. Hollywood, like Bollywood or our Cinecittà, bet everything on this. But can modern cinema really go on like this? Or, at least, do we really want to lose a place for fun, socialization, so important as the cinema was in the last century?

A brief history of modern cinema

When Rossellini was at the “Modern Cinema”

I remember fondly when my father told me about the “Patini” cinema in Castel di Sangro, our hometown, in the days of his childhood. Born in 1937, immediately after the World War II the situation in Europe was anything but rosy. There was so much to rebuild, we lived in a climate of constant tension due to the Cold War.

The United States was the only power in shape, and mass cinema could only come from there. True, we were in the McCarthyism with not very clear rules. This name comes from Senator Joseph McCarthy, who persecuted and imprisoned hundreds of progressives in 1953/54. In a climate of collective political hysteria, every innovative idea was associated with the Communists. Characters such as actor Charlie Chaplin, directors Jules Dassin and Joseph Losey were forced into exile. The rebels, out of all the “Hollywood Ten” (10 screenwriters who appealed to American freedom of expression), rightly ended up in the cage. Walt Disney and others escaped by denouncing colleagues… If you can dream it, surely you can (which then, this phrase is from Imagineer Disney Tom Fitzgerald, not Walt as is commonly thought).

The cinema of the United States to conquer the world

Despite this, America came out with its head held high. The history of cinema teaches us that the major film companies joined forces under the MPEAA (Motion Picture Export Association of America) to subdue opponents in the international market, and the government of the country led by Roosevelt first and Truman then, took advantage of its hegemonic power to circumvent foreign protectionist laws.

But was it all easy? In fact, no. In any case, the public had to be convinced, it was necessary to give them serious reasons to reach the theaters TV grew in importance every year, and for this reason the focus was on technological and stylistic innovation. Among these, color was a cinematic exclusive. This Italian article by Domenico Palattella is interesting, from which I quote:

Color must in fact be understood as the natural evolution of black and white and of Cinema as art, because if Cinema is the art of reality, color also had to necessarily rise to enhance the emotions that only this Art can offer in a way so penetrating, yesterday as today.

Domenico Palattella

Color films increased from 20 to 50% of productions in the early 1950s, becoming almost all in the 1960s. The Kodak Eastman Color film (and, before that, Ferraniacolor in Italy), which cut costs, came to the rescue for its cheapness; but it was only a consequence of what the market was asking for at the time.

Another innovation was the change of format. The 4:3 TV ratio gave way to the more eye-catching panoramic format. Then again, young people were drawn to the new drive-ins. There was a real competition, TV vs Cinema. The baby boom convinced filmmakers to look for every valid way to seduce young people, obviously also by producing TV series: they have always been popular. But the cinema, intended as the theater, was not left to itself.

Attendance at Italian cinemas throughout history

I was saying, my father… He was dying to go to the cinema, and Italy was not America. There was very little money, and 100 lire for a child was really a lot. So he saved himself, and yes… Sometimes you could access the Patini Cinema for free by making eye contact with the owner. But what a desire to go to the cinema!

Is this still the case today? How is the attendance rate of cinemas around the world evolving? For Italy, Istat comes to the rescue. A 2018 report photographs the situation from 1993 to 2017. In this period, the percentage of people who went to the cinema more than 7 times in a year has dropped by almost 20%. On the contrary, the percentage of those who went at least once, has risen. Maybe, only to remember if the place hasn’t changed a bit. This increase was helped by the increase in the female audience (which, strangely, previously seemed less fascinated by the big screen).

Other important data? More and more over-65s go to theaters: from 6.3% in 1993 to 15.2% in 2017. Positive or alarming figure? I’m honest, I can’t convince myself of the answer. To close this quick overview (if you’re curious, read the report), the main reason why you don’t go to the cinema is (coincidentally): “I prefer to watch movies at home”.

Modern cinema costs too much

And unfortunately it is boring now.

But let’s go back a little. Let’s enter our virtual time machine, and let’s go into the first half of the 1900s. Even in Italy, in 1940 on average each inhabitant went to the cinema 8 times a year. In 1950 thirteen, to reach a peak of about 17 times in 1955. From there, a steady decline. 10 admissions per year in 1970, 4 in 1980 to stabilize at around two or less in the last 30 years. Causes? TV (and to follow the internet), but also ticket prices constantly increasing until 1990. And, I might add, lack of real innovation in an increasingly innovative world. Don’t get me wrong: in production there are, and many. It is in the fruition that innovations are lacking, and perhaps it is even more important.

Cinemas in the United States from 1930 to today

A look beyond the ocean? The US market is certainly more authoritative than the Italian one, not least because of the numbers. But it is no different… At the beginning of the 1930s (about 35 years after the opening of the first movie theater, in 1894), over 70% of the population went to the cinema at least once a week. 70%! This percentage dropped to around 40% in 1935, to then exceed 60% again in 1944. Since then, a continuous loss of spectators: in 1950 it was 30%, in 1960 less than twenty. And it was the 1964 the year that, in the US market, marks the stabilization of regular cinema-goers at about 10% of the population.

A clear reason, even in the States, was the price of the ticket. Hollywood has always been an industry whose rulers celebrate themselves as superior to the “plebs”, no matter what you think. And in lean times, they would never have given up part of their millionaire earnings. If there were fewer spectators, the price goes up. Simple. Too bad it’s a dog biting its own tail.

Movie theaters create sociality and sharing

Money was not the only reason, however: there was also a social reason (which returns cyclically): during the Great Depression, in the 1930s, cinema was a way to escape the cold, heat and loneliness. Movie theaters brought together strangers, who one next to the other shared the only social moment available to all.

3D movies, the only real innovation in recent years

I remember the period when stereoscopic cinema, “3D films”, made a comeback the last time. Not many years ago, I was in the middle of my film academy (2011, good times). Here, it could be a little breath of freshness. But the idea was rejected, officially because it didn’t work. However, between the need to accustom the public and the time necessary to update theaters and productions, the real reason for not reaching results was (once again) the will of the cinema world to speculate on customers.

Stereoscopy, on balance, was not a way to offer users more. It was a way to dramatically increase the cost of the ticket. And I myself, as much as I loved seeing some films in 3D, often preferred the 2D version just to avoid the 12/13 euro entry. How many like me?

The final solution: unique and economical experience

So, is there a solution? In my opinion, yes, and there are those who support me with more authority. Well, I get off the pedestal: I support him. I will tell you about it in the next few days, my break is over, but it is basically a question of returning to making cinema a unique, unrepeatable, fun, social and, last but not least, budget experience.

Do we want to reform cinema? Let’s make a revolution together? Great things always start from the bottom… And we are in the period when it is easier to do so.

Article F.A.Q.

L'articolo Innovating modern cinema è apparso primo per Future of Cinema - by Dario Riccio.

I decided to keep the standard WordPress title … Because I think it is the most suitable to crown this first article of a blog that, hopefully, will be able to improve our technical and managerial skills.

For the moment, I have decided not to advertise these lines too much on social networks, nor on external sites. If you are here, it means that you have had the bad luck to know me personally, or that for some reason Google or another unknown search engine has thought of pointing me out as suitable to help you.

Ok gentlemen, I think I will need more help but let’s proceed in order. I will not make an editorial plan, I will not do regular publications, I will respond to your comments because I want to know you (why don’t we start right away, write down in the comments where are you reading from ).

Who is this Dario Riccio

As you may have guessed, my name is Dario. Thirty-one-year-old millennial, I grew up in Castel di Sangro, in Abruzzo, and for almost three years I have lived in Martigny in Switzerland. Of the reasons why I decided to emigrate, we will certainly have the opportunity to talk between one article and another. And of me too, I’m not attaching my CV.

For most of my life, and still today, I have been a videomaker. I currently drive buses for a living, and am passionate about cinema, business and technology.

I didn’t say expert, mind you, but passionate. I tell you this because these will be the topics that we will deal with most in this blog. Maybe a few trips every now and then, because it’s beautiful.

Now I am looking for a royalty-free image suitable for this article on Pixabay, while I wait for RunCloud and Let’s Encrypt to agree to get me an SSL certificate (otherwise you will be afraid to visit me, and I understand you), and I salute you . Thanks for being here, see you soon!

L'articolo Hello world! è apparso primo per Future of Cinema - by Dario Riccio.