It promises to leave a deep mark and represent a significant turning point in the history of cinema. For a more democratic cinema, more present among the people, uniting… What was the original cinema, and what it has failed to be for so long.

In this article I want to talk broadly, technically, about the basic idea. And share with you some possible areas of using artificial intelligence to get the most out of it on an, intentionally, small budget.

We can also consider it an update, at a much more advanced stage, of the last article written in February 2023 entitled. How to Make Low Budget Movies with Artificial Intelligence – First Steps. .

The revolutionary impact of Artificial Intelligence in cinema

At one time, special effects and narrative techniques were the magic wands of cinema, but today artificial intelligence is playing a revolutionary card. It is an incredible help, an accomplice that opens the door to unprecedented innovation. We are riding this wave, with AI by our side lending a hand in creating compelling stories, digital characters that feel real, and music that gets right to your heart. But let us not forget the human touch, the real beating heart of each of our creations.

“Artificial intelligence and cinema” is not a token motto, but the beginning of a new chapter in storytelling. We are ready to prove that the future of entertainment is no longer a distant dream: it is here, and it is animated by artificial intelligence.

Luna Dolph, Kyle and China: from virtual to real life

The story, scripted by Gérard Bernasconi , starts with the story of Luna Dolph, Kyle and China; not mere characters, but three-dimensional avatars breathing life into a fascinating and complex virtual world. These avatars are not only protagonists of a story, but symbols of our age. Vivid representations of our progressive immersion in the virtual universe, also known as the metaverse. This concept, once relegated to science fiction, is rapidly taking shape and influencing diverse industries, from video games to social networking platforms, transforming the way we interact, work and have fun.

Virtual Reality and Digital Identities

Our goal is not only to tell a compelling story, but to invite the audience to reflect on the nature of reality in a digital age. These avatars, with their complex interactions and intricate worlds, represent the ongoing fusion of our physical and digital lives. A convergence that is redefining the very meaning of identity, community and belonging.

At the core of our narrative we find Luna Dolph, Kyle and China. They are three three-dimensional avatars whose lives take place in a virtual world of extraordinary beauty. While existing in the digital ether, their story is a bridge to reality, a means of reminding our viewers of the irreplaceable value of human interaction and real life. In an age when digitization has transformed our ways of connecting, our narrative aims to use technology not as a refuge, but as a springboard to rediscover and reinvigorate authentic and tangible sociality.

Technology helps you live better

As Luna, Kyle and China navigate a fascinating metaverse, their experiences and challenges resonate with a clear message: technology, however advanced, is a tool that, when used wisely, can enrich but not replace the warmth and complexity of human connections. Our goal is to bring out the realization that despite the attractiveness of digital, real life takes place off the screen. In the shared laughter, handshakes, hugs and spontaneous moments that make up the fabric of our existence.

With this storytelling, we aspire to inspire viewers to lift their gaze from their devices and re-immerse themselves in the real world, enriching their lives with authentic experiences. Through the exploration of virtual worlds, we want to celebrate and promote the beauty and irreplaceable importance of real life and human sociality.

The film will represent the first outing in the real world for the three main characters.

How we use AI in film production

We are still in the pre-production stage, so from a practical point of view I will keep you updated in the coming months. For now, we have a rough idea: we have selected a compendium of the latest artificial intelligence (AI) technologies that are both affordable and available to all.

The film will be in fulldome, immersive 360×180-degree format. We will basically project it into domes and planetariums. This is a crucial innovation looking to the future as we push more and more toward total immersiveness. And we have a limited budget, let’s say between 10 and 20,000 euros at a glance. The film will last about 40 minutes, and will be about 30 percent in the virtual world of Luna and her friends (entirely recreated in Unreal Engine ), for the remaining 70% in the real world.

Using Luma AI for the cinema of the future

To begin talking about this technological arsenal I mention the following Luma AI, a pioneering solution that completely revolutionizes the generation of three-dimensional environments. Luma AI allows us to reproduce real environments within Unreal Engine, in photorealistic quality and even with an iPhone. Employing advanced technologies such as the Neural Radiance Fields (NeRF) and the brand new Gaussian Splatting (the latter published just three months ago by the University of the Côte d’Azur, France), we can capture the complexity and richness of the real world in digital format, put it into Unreal Engine (including the ability to move freely within the scene) and bring to life scenes previously relegated only to large budgets.

We lower the costs associated with creating detailed settings and complex objects, as well as streamline and speed up the production process. Luma AI not only democratizes access to world-class technologies, but also gives us the tools to experiment at a speed that was previously unimaginable. And it allows us to impart an unprecedented level of vividness and depth to our scenes.

Skybox AI for cinema – Simplified lighting on Unreal Engine

Another crucial tool in our repertoire is Skybox AI by Blockade Labs , for creating immersive skyboxes. Enriching virtual scenes with vital details about lighting and setting.

A skybox in Unreal Engine not only provides visually convincing surroundings (mountains, sky, distant houses, etc…), but also affects the overall lighting of 3D assets within the scene. This is what interests us most in filmmaking: it acts as an ambient light source, reflecting its colors and hues on objects, helping to create a consistent and realistic atmosphere. For example, a skybox depicting a sunset will infuse warm orange and red hues on the scene; while a night skybox will provide a cooler, dimmer light. This process helps integrate 3D assets into the surrounding environment, making the entire visual experience more immersive and coherent.

Aspect that proves essential in creating 360-degree fulldome environments. Every tiny detail is critical to sustaining the illusion of a fully immersive world.

Using Kaiber AI for our film

Kaiber AI is useful and quality, but I have a serious doubt: I don’t know if it works with fulldome video. I haven’t had time to experiment with it yet, but it will possibly help smooth out the scenes so that the assets can be merged even better. It works through AI directly on the final video files.

Artificial intelligence and cinema: Reverie AI

In anticipation of the launch of Reverie AI , we are stimulated by its promises to create virtual worlds for Unreal Engine almost by “copying” a pre-existing picture. Its ability to generate scenery that faithfully mimics reality, combined with its potential in color correction of virtual scenes, opens doors to unlimited possibilities in visual storytelling. Reverie AI promises to be an excellent complement to our workflow, improving visual consistency and ensuring smooth and convincing transitions between the virtual and real worlds.

Move.ai, cheap and working mocap

Another “smart” tool we plan to use for our film will be Move.ai, specifically the inexpensive service Move One (on launch offer at $15 a month, then expected to cost $30). With a simple iPhone app, we will be able to create simple, ready-made custom animations without too many fixes or clean-ups. This reduces time and cost, allowing you not to be limited by the animations already available in services such as Mixamo , ActorCore or the MoCap Market by Noitom.

Here a very quick video test:

Clearly, they also have the more expensive professional service that allows up to 8 rooms to be used. But we do not count on using it for this first production unless there is a very real need.

Speech-to-speech for film dubbing… Is it possible?

In our production, innovation does not stop with the creation of the digital world; in fact, we also want to take advantage of the ongoing technological revolution in traditional aspects of filmmaking, such as dubbing.

The voice of Luna and the other 3D avatars, must be consistent (not to say identical) in all media, both in the virtual and real worlds. We face a complex challenge: maintaining the same voice across multiple platforms, including social media and especially real-time streaming, without tying ourselves to a single voice actor. This allows us to have flexibility in storytelling and to adapt to various formats without depending on the availability of a specific actor.

The idea is to replicate the voices of some real actors and associate them with avatars (Luna, Kyle, and China for starters), then transforming any actor’s voice, in real time, into Luna’s, to use it for both dubbing movies and social content (as much in the original language as in translations), and for the live streaming superimposing these entries on the animated avatar in motion capture.

From the excellent but expensive Respeecher, to the ambiguous Voicemod

We explored options such as Respeecher , an advanced speech-to-speech conversion tool, but the costs for real-time are prohibitive. We’re talking about 1,000 or 2,000 a month for a few hours of use. Voicemod presents itself as a cheaper solution because of its AI voices, although there are conflicting rumors about its reliability (some even consider it to be malware or cryptojacking…). And there remains the problem of not having the rights to the voice, which they own, which will certainly prove to be a problem in the future. I do not yet know the cost for real-time conversion of Resemble.ai , which I have used in the past for the much cheaper text-to-speech, and of Veritone Voice .

Another tool that I have not been able to test is Voidol 3 , at a cost of about $300. I couldn’t find a demo version, but I admit I didn’t even try that hard to request one. It is one of several Oriental software created because of the typical Japanese passion for the anime world. But actually adaptable to our purpose, as we will see in a moment with another Japanese.

MetaVoice Live, Mangio RVC Fork and W-Okada Voice Changer, free and open-source

The final solutions, after much research, are. MetaVoice Live and the Voice Changer from W-Okada. Both open source, which reassures us that we can base the “future life” of the characters on these services, and both free. I cannot fail to mention the YouTube channel AI Tools Search which has been most useful to me. Among the most interesting videos in this area, certainly this one:

I particularly like MetaVoice: it is under heated development and has a cloud version for non-real-time conversions that provides greater quality and flexibility. This one is not free, but the cost between $10 and $25 per month can all in all be dealt with.

In contrast, W-Okada ‘s Voice Changer has many independent developers developing solutions compatible with it, such as GitHub user Mangio621 who created the Mangio RVC Fork , a software with Web interface that can transform the voice of a real actor into that of the chosen voice model. Many of these models, especially famous people, are already available on sites such as Voice-Models.com in “.pth” format. A classic format used in machine learning), but the best thing is that with the same Mangio RVC we can perform training, or training, of a custom voice. All locally thanks to a good video card. And therefore clearly always available, and for free.

I will do a specific article on this shortly, however. So you will follow me as I do some interesting tests.

Generating video with artificial intelligence

One use of AI in film production may be text-to-video, or video-to-video. But how useful can generating videos with artificial intelligence be? I mean right from scratch, describing to the AI in text (or with a very simplified video reference) what you want to achieve. Somewhat the future, to date tools like Runway Gen-1 , Genmo AI , o Moonvalley AI are little more than experiments. Useful at some junctures, but far from the quality and realism needed for a film production.

Evidently, we will have to work for a few more years to get our films

Canon EOS R5C and Dual Fisheye lens for 3D fulldome video

What about live action filming? In our journey, we are trying our hand at a very exciting combination of equipment. We always keep the goal in mind: little expense, much return. I had originally planned to shoot everything in full 360 degrees, also to make it already compatible in case of future virtual reality porting. But the cost became prohibitive, both for the camera (a professional camera like the Insta 360 Titan , which by the way has not been updated for years, costs more than 17,000 euros), and for the difficulties of starting from the very beginning with such a wide field.

So the idea is to produce the right video for the dome, in 360×180 degrees (basically half the sphere). And the lens Canon RF 5.2mm F2.8 L Dual Fisheye , paired with the Canon EOS R5C , should prove to be a winning choice. This setup not only provides immersive images, but also allows us to experiment with stereoscopic shots, which add an extra level of depth and realism.

The Canon EOS R5C camera, with its dynamic range between 11 and 12 stops and 8K sensor, offers excellent value for money. This is an important consideration for us, as we are trying to maintain a balanced budget without sacrificing quality. We will also consider whether to rent it … From Adcom , lens and camera are offered at about 200 euros daily.

To be honest, we can’t fully exploit the potential of stereoscopy yet because of projection, but it is definitely something we would like to explore in the future. It is one of those things that looks really cool and could add a special touch to our project. Have you ever projected stereoscopic fulldomes or do you have any suggestions on how we could integrate it into our work? I would be happy to hear your thoughts and ideas.

And if it will be lacking in anything compared to more emblazoned rooms…. Again, we count on leveraging AI for video quality improvement. But research, in the field, still needs to be done.

Conclusions

In the end, our choice of digital tools and equipment reflects a desire not to compromise on quality while keeping an eye on the budget.

In short, we are creating something beyond traditional cinema. Thanks to artificial intelligence and cutting-edge technology, the “cinema of the future” is no longer a dream. It is real, we want to see it happen. And with the genius of Gérard Bernasconi to the screenplay (who also gave us a great technical contribution), the precision of Michela Sette in the role of VFX Supervisor and the creativity of Michele Pelosio as director, we are forging a revolutionary cinematic experience. Get ready, because we are about to take you to a world where cinema and reality merge into a transcendental experience

L'articolo AI in Film Production: How to Produce an Innovative Low-Budget Film è apparso primo per Future of Cinema - by Dario Riccio.

A beautiful mix of cinema and theater, which has been enjoying success since 2007. We will talk about the importance of sociality, how Secret Cinema works, the importance of finding new ways to revive movie theaters, and many other topics.

Link to article: https://www.darioriccio.com/secret-cinema-immersivo-ma-in-compagnia/

L'articolo Secret Cinema: immersive, but social è apparso primo per Future of Cinema - by Dario Riccio.

In his voice we can hear, without a doubt, a passion for the new way to watch movies, with an immersiveness beyond any level ever achieved. We could say that they have “recreated” the movies in the real world.

Let’s try to understand it a little better with himself. So here is the interview with Andrea Moccia.

Introducing Andrea Moccia

Dario:

Andrea Moccia is a veteran in innovative cinema who has been trying for years to innovate and change a bit the way cinema is enjoyed.

Andrea:

Hi Dario, nice to meet you. I am very glad to meet you.

Dario:

How are you? All right?

Andrea:

It’s all right, thank you. Here in London it is gray as hell. Today there is a fog I have never seen, it feels more like Switzerland than London.

Dario:

Okay, a Londoner telling me about the fog like that is interesting. People say it’s always there, but I’ve been to London a few times actually, and I’ve never caught it with fog….

Andrea:

Fog not too much. The sky is always gray. But fog not too much.

Dario:

Yes, indeed the sky was gray. I clearly remember that. So, tell us a little about you… You’re from Rome. So a pure Italian, from the capital. Moved to London since? What was it like? Why did you go to London? How did you find yourself there?

Andrea:

I moved to London in 2010 for studies. I graduated here, and during my last year of University, I was very lucky to be introduced to the world of Secret Cinema. I went to a couple of their shows, and among them there was one based on The Shawshank Redemption. After going to this show, I was absolutely impressed by this world, this format. And I found a way to start working with them.

At first as a trainee. While I used to study in the mornings for exams, I went to the office in the afternoons. Then I started working with them, first as an assistant and then for the last five or six years as a producer.

Dario:

Okay, I just stop you a moment, just to get a good understanding. First of all, I presume you were studying something related to cinema. What were you studying?

Andrea:

No, I did business administration and management. So not directly related to cinema. However, it is a degree with which I found many affinities as a producer. Let’s also say that I’ve always had an interest in art, in performance. When I was a kid I did some acting, I liked that world. And I found a way to apply my degree to the world of performance and art.

How Secret Cinema is interpreting immersive cinema

Dario:

OK. Because in fact Secret Cinema, even to explain it a little bit to our listeners, what is it? Very broad question.

Andrea:

Of course. So, Secret Cinema is a company that specializes in what we call “immersive cinema”. And what the company does, is basically take these big abandoned buildings, or big spaces, pick a movie and rebuild the whole set and world of the movie inside this building.

Then spectators buy a ticket. The moment they buy a ticket, they are assigned a character. We tell them how to dress and who they are within the world. Never anyone among the main characters. Always characters created by us who could exist in that world.

Dario:

Extras…

Andrea:

That’s right, extras basically. And people come to experience the world of this film. We have covered great classics such as Back to the Future, The Battle of Algiers, Star Wars. Up to much more niche films, for example Park Chan-wook‘s The Handmaiden, or I, Daniel Blake.

The company was founded in 2007. It operates on a global scale at this time. We have performances in the United States, England and China.

England, the ideal country to grow up

Dario:

Basically, however, you do most of the shows in London. Or they are somewhat evenly divided among countries.

Andrea:

Most of the shows we always did in London, yes. Let’s say London is our base, where we grew up. And England has been crucial to that growth, I think.

In terms of bureaucracy, but also as theatrical industry, there is already a base where something like this has been able to grow far more easily than in other countries. For example, in my opinion even if the idea had started in the USA, it would have been more complicated. Even just to access these buildings, which are usually large mansions not built to accommodate thousands of people for an event.

They can be abandoned factories, or warehouses, things like that. So the process of getting permits, the process of turning these buildings into what they are, then assets or spaces for very large events, is very complicated and expensive.

And I think, in the early days, these ingredients could have worked only in England.

Close cooperation with film companies

Dario:

Do you guys have any help, any cooperation from the film companies, or do they not care? Do you just buy the rights to reproduce the film?

Andrea:

Look, it really depends from movie to movie. There are some properties that are perhaps a bit more dormant, with titles that are not being redeveloped or, otherwise while having historical value, are not bringing in much revenue at this time.

At that point maybe the studios contact us and ask if we want to do something with these films; in short, they want to generate new interest. Then, of course, there are other films where we ask for and pay rights to the studios, working with them as licensees.

There is always collaboration at the operational level, though, never just at the financial level. There is always an involvement of “Studios” and creators in our creative process.

Dario:

Okay, so they are interested anyway. It is also a way for the companies to promote the film.

Andrea:

Absolutely. We make many films in years when a sequel is released. For example, Blade Runner, which we did in the year that Blade Runner 2049 was released. Or Star Wars in the year the franchise started again. So yes, many times there are also marketing interests behind it.

A beautiful thing Baz Luhrmann said to Fabien (Fabien Riggall, nda), the person who created Secret Cinema in 2007, after coming to a Moulin Rouge! show, was:

I made the movie, but you created the world.

Baz Luhrmann

And that, I think, is a very appropriate summary for what we do. We create a world of the film.

How to choose films suitable for immersiveness

Dario:

You are currently working with Marvel, and then with Guardians of the Galaxy. For example, who wanted to make this film that is now on the schedule? You asked Marvel or it was Marvel who asked you to develop the world, the marketing.

Andrea:

Guardians of the Galaxy is the last show that I worked on at Secret Cinema. I recently left Secret Cinema.

In reality, it is originated from a much larger collaboration between Disney and Secret Cinema. We have a contract of several years with Disney in which we have access to several of their titles. I mean, who wouldn’t want to work on the Marvel world? Who wouldn’t want to join the Marvel Universe…

Dario:

I guess.

Andrea:

Basically when we choose movies, we don’t always choose them because it’s a great movie or a great story. There are many beautiful films that do not work for a Secret Cinema, precisely because we have to create a world, we have to give viewers the opportunity to disguise themselves, to impersonate someone.

Guardians of the Galaxy, being all set in space with this mix of alien races, we felt it was the most interesting world we could recreate in a physical, real-world setting.

Whereas instead, I don’t know… One of my favorite Marvel movies is the first Iron Man. However, it does not offer many insights to engage viewers. While instead you come as an alien, to watch Guardians of the Galaxy. So, all the people come in makeup and in these beautiful, exaggerated costumes; and part of the beauty of coming to a Secret Cinema is just that, right?

There are people who spend weeks, months, creating costumes for themselves, putting all the photos of the process on Instagram. Many of our fans are also cosplayers, or people who work in the creative industry. It is also an opportunity for them to show off their creative talents in a social, sociable environment.

The typical structure of a Secret Cinema

Dario:

Oh yes, because the Secret Cinemas, practically, how are they structured? Is there the screening of the movie in costume, or you see the movie and then move on to other environments…

Andrea:

In a typical Secret Cinema, we sent you basically an email. The email redirects you to a website. On this site we usually have personality questions, questions that will get you assigned a character. Each character is part of larger groups, and each group has its own specific mission; its own specific path within the show.

Then, when you get to the show, the first two or three hours are like an open world video game, where you can go and do various missions; talk to all the characters. You go to sets, you can get drinks, you can go dancing, there’s music, there’s food… Yes, it’s like a video game where you choose your own adventure.

Dario:

A real video game, in short, rebuilt in reality…

Andrea:

Correct. So this is the beginning. Then, after two or three hours, you watch the movie. And during the film we also reenact many live scenes. So, for example, if you’re in Back to the Future and there’s the DeLorean leaving and going to 1986, there’s also a DeLorean going next to you. Just a real DeLorean passing by you.

Or, when Doc falls off the tower, we had a Doc starting from the clock tower on a zipline, going over the spectators and landing on the other side of the field. Because Back to the Future is a show we did outdoors.

For Star Wars, we had an X-Wing that flew over a thousand viewers, landed and out came Luke Skywalker. So it’s all things like that. These are shows that reach production values that few other events can achieve.

Is VR the future? Or will we get back together again?

Dario:

Of course, the parallel with virtual reality comes to mind. In the sense that you have similar experiences, with the major difference being that you are in a sociable environment, with others, and not alone locked in an headset. And this could also make a big difference.

In your opinion, or even according to people you have talked to (viewers, colleagues…), how will this situation evolve? Will people always want to be with each other?

Andrea:

I, at least, am an absolute fan of shared experiences. For me, there is nothing more beautiful than going to a concert with thousands of other people; or going to a movie or theater.

There is just a feeling that you can’t beat. Unless virtual reality will reach levels we cannot even imagine at this time. I don’t think it will come to replace the live experience.

What we are trying to do is to put some magic back into going to the cinema, right? Fabien, when he started this concept of Secret Cinema he was always thinking about the days when people went to the movies dressed up; when going to the movies was an event, when it was really a celebration.

This is what we want to report. It came into being almost a reaction to these multiplex giants. These multiscreens where the experience of going to the movies has been somewhat “formatted,” and the magic has been lost a bit. So this is what we are looking for. And this, in my opinion, is the challenge that cinema, and other art forms as well, will have to face.

But how do you do it? How do you put this magic back? How do you go about breaking a format that has come to produce this mechanization of entertainment? I don’t know how to explain…

So many seek a new way of conceiving cinema

Dario:

Yes, everything became very standardized. In fact, this is something I am glad to hear you say. Because I also support it, personally and in my blog. In mindset I am like that. I also mentioned you, in the email I sent you, that I follow a lot the studies and life of Douglas Trumbull, the creator, the maker of the special effects of 2001: A Space Odyssey.

Andrea:

But also of Blade Runner. I think he died recently, Douglas Trumbull.

Dario:

He died very recently, this year in February I think. He, however, has spent this life trying to find a solution. Obviously its was not Secret Cinema. It was something less “scenic”, but still perhaps more reproducible. That’s the difference.

Secret Cinema I think is a wonderful thing. I still don’t, I just met him recently, but I will definitely have to participate because it is beautiful as an idea. It is very expensive, though; organizing a single screening will cost impossible sums of money for a small business to handle.

Andrea:

Absolutely. However, there are manners. The shows they are doing now are shows of ten million and up per show. So very, very large productions, with very real economic needs. You have to sell hundreds of thousands of tickets to make these things work.

Skates and social issues: where Secret Cinema started.

But let’s remember where Secret Cinema started, though. Because Secret Cinema started with a screening in an abandoned skateboard park, where they basically projected that Paranoid Park on a sheet. And two hundred people came.

So there are simple ways to do things in an effective way, especially when you have films that maybe deal with serious and difficult issues. At that moment you don’t really want to go and recreate the film, but you want to bring the theme to light more. And so we also do a lot of social things.

For example, we did screenings against censorship when films were censored; then screenings in refugee camps. Cinema is not just entertainment, is it? There is a whole sector of cinema that is about entertainment, fun, and it is beautiful that it exists. But there is also a whole part of cinema that wants to send strong messages.

And that in my opinion also needs to be reinvented. Spaces must be found. I am a huge Marvel fan, but you can’t just go to see Marvel at the cinema. Other kinds of films should be seen as well, because there are beautiful things out there.

Dario:

Yes, and then cinema almost was born for that. Well, apart from the very first screenings, the development of cinema internationally was because of this. To pass messages, at the time perhaps more political than anything else, which then quickly became moral or various messages in the early decades.

Andrea:

But we may have forgotten that a little bit.

The importance of moral awareness in cinema

Dario:

Effectively, yes. Oh God, in part. From one point of view, even just many movies or series on Netflix still try to maintain a moral. Then, currently, even those have become standardized.

In the sense they go by fads. At this time I personally notice feminism or LGBT+ is trending; there is an attempt to raise awareness of these issues. Sometimes there is the topic of animals, meat, vegetarians. However, we often talk about fads that are then forgotten. When, instead, perhaps it should be more consistent.

Andrea:

Yes, it is true. And in my opinion, here again we come back to the issue of commerciality. If people don’t go to the cinema, don’t buy tickets, unfortunately these films will no longer be made. Ways have to be found to bring people back to the very act of going to the cinema.

I have been following Cinema America in Rome for many years, and what they have been doing: bringing cinema to the streets, and various initiatives just to rekindle the passion for film, to go to the cinema and watch something all together. Which I think after two years of pandemic we really have to “have it taught” almost, don’t we? People are perhaps still a little afraid to go out, to socialize and are definitely more cautious. However, there are ways to continue living without taking risks.

Dario:

In past years I have organized various outdoor cinemas around Italy. We have done cinemas in Tuscany, in Montalcino, in Ladispoli, and also in Rome, in various places around the capital, in Abruzzo… I mean, in different places and basically they were all very, very well attended.

Of course, they were free cinemas; they were almost always contributed by municipalities or associations. So there was no ticket clutch. But people liked them; they went there; they spent evenings together and they were all super happy to do something different.

The problem is that, precisely, they were happy to do something different. Instead, it should become the norm. I remember my father, born in 1937 (other times…) telling me about when he was a child, or at least a teenager, and the movie theater was kind of their bar. In the sense that they went to the movies almost every day. It cost very little, and they went there to be together and spend two hours. I mean, it was just different culture from this point of view, but then it evolved.

The death of the high street

Andrea:

And maybe that’s part of the solution too, right? The Internet, new technologies, have changed the way we do everything. For example, here in England we are experiencing a time when there is a lot of conversation around “the death of the high street“, the death of the street with all the stores. And how can we reinvent stores so they don’t die? How can we reinvent the high streets?

Stores are reinventing themselves. Here a clothing store, is now no longer just a clothing store. It is a clothing store with a cafe, with a bookstore. A bit of a gathering place. And perhaps cinemas also need to reinvent themselves and become gathering places. It would be great if, for example, all theaters had a small library on the history of movies…

I mean, in my opinion, you have to detach yourself a little bit from this format of cinema. You go, you get the popcorn, you get the Coke, you sit down, you just leave. If cinema is turned into an experience, if cinema is turned into a reason to go out, to socialize, in my opinion it will find its place again.

Dario:

Definitely. Experiences also and especially not reproducible at home. Because the problem with current cinema is that it is reproducible almost entirely at home, with a video projector or a simple TV. You can eat popcorns, you can drink Coca-Cola or beer. I mean, it lacks perhaps that extra reason that makes you say: I can only do this thing there. And we all are lazy, basically, as the human race.

Andrea (smiling):

Me first.

Dario:

Me too, don’t believe. And so, rightly you stay home. But yes, there should be one more reason.

Multi-sensory cinema: yes, but without distractions

Let’s talk about one last thing, then if you want to add more, otherwise we’ll end since we’ve already been half an hour together. I wanted to ask you, what do you think about using more than two senses at the cinema? Have you ever experimented with combining smells, possibly flavors, so a combination with foods or other things but always related to the film at the cinema? Also touch, albeit more complex (although in Secret Cinema there is actually touch, you can touch things of the film as well).

Andrea:

It is a trend that is going a lot, for example in Asian countries. There are some big companies that have started to make these really big 4D cinemas where they include “smell,” water splash, something like that. Or there are other cinemas, especially here, where for example during a screening they bring you food or drink.

So there are already things that unite them. For me, the question one must always ask himself is: how do these things go to enhance, to improve, the viewers’ experience? If these things are done with the intention that maybe a certain smell can improve my understanding of the film, of what I’m going to see, offering a new perspective, then I’m absolutely pro. If these things are just being done to do something new, then I think they can occasionally have the opposite effect. Becoming distractions rather than a magnifying glass.

Dario:

Okay, you say that they could become something that makes you lose the sense, the lesson. It goes back to the argument we were making earlier.

Andrea:

Correct. However, I am always “pro” experimenting, I am always “pro” trying new things. New things need to be tried, and they also go in this direction. Absolutely. However, as long as the intention is there. Now I can’t give you any concrete examples, but there must be a reason behind it. There must be an intention.

Dario:

Yes, I get the concept. Indeed an example is complex so, on the spot. It depends on the particular story, the single film, the single screening. It is a beautiful answer that makes total sense, though.

Because it is true that there are often so many things, or so many technologies, only done because you had to find a problem to the solution; not the solution to the problem. Which by the way, at the moment, is virtual reality: a totally useless technology the way it is structured that they are trying to find a use for.

Technology should help, not destroy

Andrea:

Yes, let’s say I am a bit, um, technophobic on these points. I really like analog. I’m one who read books, who have an agenda. In this respect I am a little behind.

There are some really cool virtual reality applications. There have been exhibitions here that I have gone to where it has been used really well. As there have been other really low-level things.

Dario:

In which a television set was fine.

Andrea:

That’s right, let’s say. In my opinion, the technology is still a few steps behind the ideas.

Dario:

It is true, exactly that. It goes a long way perhaps in gaming, because still we come back to the argument that spectacle, the visual of images, of the location, and the involvement do so much. But it’s the only area currently that’s really going… With education they’re trying to do something, they’re inventing, but we have to wait to really have value added to the traditional method that’s always been here, and maybe can be modernized with little.

Andrea:

Correct.

secretcinema.org, to learn about Secret Cinema

Dario:

Okay Andrea, I don’t know if you want to add anything else.

Andrea:

No Dario, thank you very much. It was a really interesting talk. Go to the Secret Cinema website (www.secretcinema.org, nda), and go see what it is. If you haven’t been, absolutely try to see it, because that are truly unique experiences that will certainly bring you closer to whatever movie you go to see at Secret Cinema.

Dario:

Okay. I also support the invitation; I have already gone to see the site and from that I want to move on to see the real Secret Cinema. So, I will have to go to London as soon as possible to do that. It’s really beautiful. An innovation like few in the world, and as such it deserves to continue.

Also to you Andrea good luck with everything. Directly for your career, your future, and any other innovative experiment you get involved in.

Andrea:

It’s going to be a very, very interesting year, I think. There are many interesting insights, so definitely maybe we’ll have a chat later on my new adventures.

Dario:

Definitely. That’s all I’m waiting for. Thank you, Andrea. Bye!

L'articolo Secret Cinema: immersive, but social è apparso primo per Future of Cinema - by Dario Riccio.

Articolo correlato sul mio blog: https://www.darioriccio.com/che-cosa-cinema-esperienziale/

L'articolo Che cos’è il cinema esperienziale è apparso primo per Future of Cinema - by Dario Riccio.

Current cinema, a modern derivation of the ancient analog, has five characteristics: static and linear structure, dual mode, episodic, mainly in third person and not interactive. We will see later what each of these voices means, while I continue to think about what the cinema of the future will be, about immersive cinema, and how to make a good prototype.

I started this article on a Swiss train, and finished it at Barcelona’s charming Espai Joliu. I love trains, being able to work or relax while moving between two locations. Among my corporate benefits is the general season ticket for all means of transport in Switzerland: my luck. A few days off from work, I take the opportunity to reach Domodossola (the first Italian town across the border), or wander the length and breadth of the country of the Helvetians, which reserves surprises and magical corners almost everywhere.

How important is immersion in the cinema of the future

I am part of the line of thought, now increasingly in vogue, which imagines a single evolution for cinema: immersive cinema. According to purists, or “traditionalists”, cinema must be left as it is. But we have already seen that, after all, they are not necessarily right.

As you may have noticed, I am not at all cadenced in publishing articles on my blog. I don’t like online editorial plans, I prefer quality over quantity, avoiding in-depth articles only to come out “Friday at 7”. This article took me days of research, maybe more, and clearly blogging is not my primary activity, but it has resulted in a roundup of ideas and characters that I hope will be a definitive answer to the question: how will the cinema of the future be?

Few have seriously tried to innovate it, and in this article we will get to know some of them. Certainly not for lack of a market. Rather for lack of resources, lately more and more projected to the lucrative, and more manageable, virtual world.

This article follows from: Innovating modern cinema

Advantages and disadvantages of modern cinema

I reflected on the advantages of cinema as it is conceived today:

• It can be used on the move, even here on the train.
• Other actions can be taken, delegating the mere role of companion to the audiovisual.
• It can be used in company.

I ask you for other advantages, I really want you to comment. Turning to the disadvantages:

• The methods of use have been updated little for decades.
• It is not able to “fill” all the senses of the viewer.

So let’s create a system, using already existing technology, able to minimize these disadvantages.

Analysis of the advantages and disadvantages of modern cinema

1) You can watch cinema on the go

Clearly, “old style” cinema by definition is destined for theaters. However, in order to recover costs, a production must be usable by as many people as possible. A good substitute for the small screen are virtual reality viewers, which allow you to view stereoscopic 360 ° videos.

2) You can do other things while watching a movie

Here virtual reality, at present, shows all its limits due to the complete isolation it creates. Bypassing this limit is perhaps possible with augmented reality, or with a serious upgrade of existing virtual reality technology.

3) You can watch a movie in company

By designing an update of the cinema system intended as a “cinema”, for decades there has been a technology used above all in scientific dissemination: the fulldome, ie projections in a dome. However, the same system is not new to film experimentation, even by great masters of the past.

4) The methods of use have been updated little for decades

The first disadvantage of cinema is rather an observation. If, on the one hand, great directors or companies have brought immense innovations in shooting methods (think ofAvatar di Cameron con una Motion Capture all’ennesima potenza, o alla Industrial Light & Magic che insieme ad Epic ha dato il là ad una seria Virtual Production), poco si è fatto per le sale.

The transition from film to digital, without wanting to diminish it, is comparable to the change of our home TV. Best quality, lower distribution costs, maximum stereoscopy but otherwise everything as before.

5) It is not able to “fill” all the senses of the viewer.

A century ago, a great innovator, today’s cinema fails to offer that “something more” compared to other more modern tools.

When looking at an iPad or modern TV, the field of view covered by the image is approximately 25°. The cinema screen, on average, covers a field of view of 50° on the 360 of the sphere (it depends a lot on the position in the room). It is necessary to increase it, if you really want to return to offer that “wow” effect, due to the immersion, which is worth the trip and the ticket.

The acoustics are already very good; Dolby Digital systems allow excellent immersion. What is missing is still the feel, the smell and, why not, also the taste.

Why virtual reality is not popular

I’ve been trying to figure out why you never want to put a helmet on, and a recent University of Glasgow paper / survey by Laura Bajorunaite, Stephen Brewster and Julie R. Williamsoncertainly helped.

The paper deals specifically with the use of VR viewers in public transport, which in itself is a place of frequent cinematic use. But the discourse is expandable, with the necessary adaptations, a little more generally.

At the base there are clearly reasons of personal safety, but also of comfort. Social acceptance is also still a long way off, making VR users “stupid” in the eyes of other passengers. Progress is still being made to make one’s virtual world more integrated with reality.

For example, occasionally throwing an eye to the real world with the cameras installed on the viewer, or having indications on the positioning of other human beings in real space, can reassure us. A complete Virtual Reality must be avoided in favor of Augmented Virtuality; as already defined in 1999 by Prof. Paul Milgram in its line of continuity from real world to virtual world.

I invite you to read the paper to learn more, but the concept we must take into account is that total immersion is a problem.

Immersive cinema

We have already covered the history of fulldome cinema in detail (article available here), but To the Moon and Beyond, is considered by many to be the first immersive film in history. The first to cover a 360 ° field of view, filmed with the Cinerama technique using a single fish-eye lens, on 70mm film at 18 fps, and projected onto the roof of a large dome about 30 meters high.

If its use was still somehow “didactic”, we moved away from the astronomical canons to bring instead a greater involvement towards the issues of transport. The film also influenced Stanley Kubrick, hiring its creators as VFX consultants for the legendary 2001: A Space Odyssey. Lester Novros and our next protagonist: Douglas Trumbull.

Douglas Trumbull, a mission: to save cinema

Douglas Trumbull was a man with a great idea:movies can be more realistic and more immersive than they already are.

I would have liked to meet him in person, he was a myth for me. I tried to write to him a couple of times, offering to visit him at his Trumbull Studios in the countryside of Massachusetts, but unfortunately I got no reply. Perhaps the addresses I had were no longer current, perhaps he was too busy as well as old. I don’t know, the fact is that he is no longer with us except through his studies that still have the time to change the history of cinema.

He was one of the greatest experts in the world of special effects, first analog and then digital. Creator of amazing graphic masterpieces in the top of last century Hollywood production (2001: A Space Odyssey, Close Encounters of the Third Kind, Star Trek …). After starting to smell the decay of traditional cinemas, he devoted his entire life to finding a solution. Which he too found in immersive cinema, as a true pioneer.

Perhaps due to his analogue habit of always finding physical and real ideas rather than creating a new app or a new system to monetize users’ time while they are alone or at home, Douglas was a multinational craftsman. He certainly had art in his DNA: his father, Donald (Pappy Trumbull to friends) had been involved with VFX since the 1930s. Not officially accredited, he worked on The Wizard of Oz from 1939.

It was during the filming of 2001: A Space Odyssey that, only 25, Doug created the legendary psychedelic color stream “Slit Scan” based on the technique developed shortly before by John Whitney for Vertigo.

Immersivity is the future of cinema

I report part of an interview made by Filmmakermagazine.com to Trumbull himself.

If you look at the industry, it is moving more and more towards immersive experiences. Take virtual reality, for example; the problem is that people don’t want to wear the headset. Immersiveness is what TV cannot offer; when 3D took off, everyone started making 3D TVs. But now nobody makes them anymore.

I think the problem was miniaturization. When you look at a 3D image on the big screen, you have a 6 foot (1.82 meters) figure standing in front of you, while when you put it on a TV screen you have like a 6 inch (15 cm) figure. When I took the projects we were working on and played them on the TV screen to see what they looked like, the effect was completely lost.

Douglas Trumbull

Magi Pod: innovating the cinema

Douglas Trumbull in his later years developed with Trumbull Studios a system that united many of the key points for the renewal of cinemas. A redesign of both shooting and projection systems. At its core, 120fps 4K projection solves a problem Kubrick already denounced over 50 years ago: 24fps blur is obnoxious and unrealistic. For this already 2001: A Space Odyssey was shot at a high framerate, although almost no room was equipped for reproduction.

Trumbull wanted to bring back the magic he himself felt in visiting the curved screens of the Cinerama in childhood. But technically there were many problems to be solved.

Troubleshoot 3D projection

To begin with, stereoscopic projection requires halving the framerate (or doubling the projectors, which would be uneconomical). This means 60 fps projection per eye.

Then there is another, more concrete problem: in 3D films both images (right and left) are recorded simultaneously. But they are then projected one after the other, alternately (right, left, right, left eye etc …). There is therefore a small difference that is not visually perceptible, but not negligible from our unconscious. Solution? Record the two images with a distance of 180 degrees / shutter, then alternately, as they will then be projected. This solves a second problem, which concerns the “cinematic look”.

The high framerate is not in fact used in cinema because it makes the product too similar to video and TV. Trumbull then concluded to increase the flicker effect originally given by the cinematic 24 fps thanks to the recording of the left and right frames of the stereoscopy with 180 degrees / shutter distance. In addition to reducing the luminance of the screen (in the Magi equal to 14 FtL, less than 50 lux), but not reducing the gain of the screen itself.

The entire Magi patent aims to increase audience involvement: it translates into projection in the Magi Pod.

A small cinema, 70 seats ll in the best viewing area; toroidal silver screen (curved both horizontally and vertically) with 3x gain to increase contrast and therefore immersion; 32 speakers for enhanced surround effect; rotary subwoofers capable of reaching 1 Hertz, creating a “4D” effect with the absurd vibrations.

Trumbull’s years of experience leading the technical departments of the biggest blockbusters, in a single, magical, immersive cinema. If knowledge must always go forward, it cannot be ignored in the design of the cinema of the future.

Cross reflections in fulldome cinema

By designing a fulldome cinema, the differences with the Magi Pod are many and fundamental. The Magi does not in fact fill a dome, but only a part of it. And you cannot simply “enlarge” the field of view to compose 360 degrees. There is a first big problem: cross reflexes.

Anyone who has worked with high gain projection screens will have realized that a silver, 3X amplified screen is a kind of mirror. Reflecting both the light coming from the outside and the same image projected from one side to the other in the case of a convex screen. A problem that also experienced Steven Spielberg’s 1991 production, Back to the Future: The Ride; fulldome re-proposition of the original Back to the Future.

By limiting the radius, and keeping the spectators within an optimal viewing angle (therefore in central positions) the problem is reduced to almost zero. But a complete sphere must necessarily be a white screen with a reflection ratio of less than 1, or the solution is still to be studied. And in this case we will certainly have to do practical experiments in the near future.

Immersion in the cinema of the future

In fact, almost every previous study we talk about in this article is in some way linked to the digital world. But, as you may know, I am a staunch supporter of sociability and the return to real life.

The room will have to resemble a space and time machine, where you will enter a totally new dimension for the duration of the screening, and from which you will come out with a complete experience. Above all, in company.

The world is heading towards the conquest of space, which will initially be within the possibilities of a select few. Others will want to try new experiences, and they will have to do so while necessarily being on the ground.

Five senses for immersive cinema

The viewer will have to be the protagonist, so the narrative will also have to change. We remember well: without disturbing the intuition, the sixth, we still have five other senses.

Today’s dual cinema manages to cover only two of them: sight and hearing. And, especially for sight, we can do a lot to improve the situation. Reading the number of Wired Italia of December 2021 (magazine that is always full of interesting ideas) I found this paragraph in an article by Henry Jenkins (lecturer at University of Southern California, director of the comparative media studies program at MIT in Boston for more than 10 years; not the last of the arrivals):

I am convinced that the metaverse will not be the future both because devices such as Oculus viewers are terrible (especially for those who, like me, wear glasses), and because it is not the visual experience that guarantees the highest level of immersion, but that audio and haptics, as evidenced by decades of studies and pioneers such as Metthew Shifrin and Nonny de la Peña.

Henry Jenkins

Jenkins’s idea, similar to mine in many ways, intrigued me to get to know musician Matthew Shifrin.

Matthew Shifrin: Legos and the cinema of the future

Matthew Shifrin is a young musician, blind from birth. Since he was a child, a Lego enthusiast, these allow him to experiment, touch, “see” without eyes. Thanks to touch, and therefore also to Legos, he can understand how the world works; as did. According to his eloquent example, he will never know what the Statue of Liberty is like without climbing it. And climbing the Statue of Liberty can be problematic… Therefore, a small Lego statue perfectly fulfills its function.

It is his thirteenth year of life. At the door rings Lilya, family friend. She approaches Matthew with light steps, looks at him and says cheerfully: “I have something for you!”. Matthew, confused, smiles at her as the woman places a package in her hands. The child opens it, touching a sheet entirely written in braille: these are the instructions for building Lego structures by himself.. His eyes shine, he is so happy. And he wants to get to work right away.

Lego for the blind

The story leads him to found, years later, “Lego for the blind“. An association, with its website, where you can download instructions like these for about thirty Lego products. And convinces the same company to include them in new products. He also created a system to help the blind to climb the rocks, really brilliant.

Definitely an interesting character; but you may be wondering why we talk about him in an article on the cinema of the future. I understand you, I was so taken by his story that for a moment I did it too. Then I remembered where I wanted to go.

In his TEDx Talk at Suffolk University, starting at minute six, he explains it well and in a very funny way.

Blind people in the cinema of the future

The point is: if the world manages to be immersive for a blind person, cinema can never be immersive in a similar or equal way to the real world if it is not for a blind person. The fact that the cinema of the future can be enjoyed in a better way also by people with disabilities, can only be an excellent and stimulating side effect.

While studying singing and accordion at the New England Conservatory in Boston, he continues to look for ways to make modern life more suitable for people like him.

In episode 3 of his Blind Guy Travels podcast, Matthew describes going to the movies with his friend Ben. The need to have Ben to describe the visual part of every single scene, and even thus not have a complete idea of the whole, leads him to create with his university a tactile device for the cinema.

With the support of the New England Conservatory’s Entrepreneurial Musicianship Department, Matt and his business partner have developed a programmable vibrating vest integrated with technology that mimics the sensations of overturning, flying and falling. This allows a new audience to effectively follow the Marvel Daredevil saga, a blind lawyer who becomes a superhero at night.

A few days ago I wrote an email to the conservatory asking for more details on this system, unfortunately without receiving an answer. I plan to comment on this article if I have any news in the future, it is interesting.

We will talk about experiential cinema

I have ready an analysis of the characteristics of current cinema, and of how they evolve in experiential cinema. How the latter is interactive, immersive, algorithmic and other important studies for the purpose of creating our cinema of the future.

But I decided to end the article here, it was getting too long and dispersed. I have already copied the text into a new WordPress post, which I will complete as soon as possible and publish.

I renew the invitation to comment, to share doubts and criticisms, and above all to participate in this innovation if you like it. You can also write to me, my address is dario.riccio@gmail.com. See you soon!

L'articolo Immersive cinema is the future è apparso primo per Future of Cinema - by Dario Riccio.