It promises to leave a deep mark and represent a significant turning point in the history of cinema. For a more democratic cinema, more present among the people, uniting… What was the original cinema, and what it has failed to be for so long.

In this article I want to talk broadly, technically, about the basic idea. And share with you some possible areas of using artificial intelligence to get the most out of it on an, intentionally, small budget.

We can also consider it an update, at a much more advanced stage, of the last article written in February 2023 entitled. How to Make Low Budget Movies with Artificial Intelligence – First Steps. .

The revolutionary impact of Artificial Intelligence in cinema

At one time, special effects and narrative techniques were the magic wands of cinema, but today artificial intelligence is playing a revolutionary card. It is an incredible help, an accomplice that opens the door to unprecedented innovation. We are riding this wave, with AI by our side lending a hand in creating compelling stories, digital characters that feel real, and music that gets right to your heart. But let us not forget the human touch, the real beating heart of each of our creations.

“Artificial intelligence and cinema” is not a token motto, but the beginning of a new chapter in storytelling. We are ready to prove that the future of entertainment is no longer a distant dream: it is here, and it is animated by artificial intelligence.

Luna Dolph, Kyle and China: from virtual to real life

The story, scripted by Gérard Bernasconi , starts with the story of Luna Dolph, Kyle and China; not mere characters, but three-dimensional avatars breathing life into a fascinating and complex virtual world. These avatars are not only protagonists of a story, but symbols of our age. Vivid representations of our progressive immersion in the virtual universe, also known as the metaverse. This concept, once relegated to science fiction, is rapidly taking shape and influencing diverse industries, from video games to social networking platforms, transforming the way we interact, work and have fun.

Virtual Reality and Digital Identities

Our goal is not only to tell a compelling story, but to invite the audience to reflect on the nature of reality in a digital age. These avatars, with their complex interactions and intricate worlds, represent the ongoing fusion of our physical and digital lives. A convergence that is redefining the very meaning of identity, community and belonging.

At the core of our narrative we find Luna Dolph, Kyle and China. They are three three-dimensional avatars whose lives take place in a virtual world of extraordinary beauty. While existing in the digital ether, their story is a bridge to reality, a means of reminding our viewers of the irreplaceable value of human interaction and real life. In an age when digitization has transformed our ways of connecting, our narrative aims to use technology not as a refuge, but as a springboard to rediscover and reinvigorate authentic and tangible sociality.

Technology helps you live better

As Luna, Kyle and China navigate a fascinating metaverse, their experiences and challenges resonate with a clear message: technology, however advanced, is a tool that, when used wisely, can enrich but not replace the warmth and complexity of human connections. Our goal is to bring out the realization that despite the attractiveness of digital, real life takes place off the screen. In the shared laughter, handshakes, hugs and spontaneous moments that make up the fabric of our existence.

With this storytelling, we aspire to inspire viewers to lift their gaze from their devices and re-immerse themselves in the real world, enriching their lives with authentic experiences. Through the exploration of virtual worlds, we want to celebrate and promote the beauty and irreplaceable importance of real life and human sociality.

The film will represent the first outing in the real world for the three main characters.

How we use AI in film production

We are still in the pre-production stage, so from a practical point of view I will keep you updated in the coming months. For now, we have a rough idea: we have selected a compendium of the latest artificial intelligence (AI) technologies that are both affordable and available to all.

The film will be in fulldome, immersive 360×180-degree format. We will basically project it into domes and planetariums. This is a crucial innovation looking to the future as we push more and more toward total immersiveness. And we have a limited budget, let’s say between 10 and 20,000 euros at a glance. The film will last about 40 minutes, and will be about 30 percent in the virtual world of Luna and her friends (entirely recreated in Unreal Engine ), for the remaining 70% in the real world.

Using Luma AI for the cinema of the future

To begin talking about this technological arsenal I mention the following Luma AI, a pioneering solution that completely revolutionizes the generation of three-dimensional environments. Luma AI allows us to reproduce real environments within Unreal Engine, in photorealistic quality and even with an iPhone. Employing advanced technologies such as the Neural Radiance Fields (NeRF) and the brand new Gaussian Splatting (the latter published just three months ago by the University of the Côte d’Azur, France), we can capture the complexity and richness of the real world in digital format, put it into Unreal Engine (including the ability to move freely within the scene) and bring to life scenes previously relegated only to large budgets.

We lower the costs associated with creating detailed settings and complex objects, as well as streamline and speed up the production process. Luma AI not only democratizes access to world-class technologies, but also gives us the tools to experiment at a speed that was previously unimaginable. And it allows us to impart an unprecedented level of vividness and depth to our scenes.

Skybox AI for cinema – Simplified lighting on Unreal Engine

Another crucial tool in our repertoire is Skybox AI by Blockade Labs , for creating immersive skyboxes. Enriching virtual scenes with vital details about lighting and setting.

A skybox in Unreal Engine not only provides visually convincing surroundings (mountains, sky, distant houses, etc…), but also affects the overall lighting of 3D assets within the scene. This is what interests us most in filmmaking: it acts as an ambient light source, reflecting its colors and hues on objects, helping to create a consistent and realistic atmosphere. For example, a skybox depicting a sunset will infuse warm orange and red hues on the scene; while a night skybox will provide a cooler, dimmer light. This process helps integrate 3D assets into the surrounding environment, making the entire visual experience more immersive and coherent.

Aspect that proves essential in creating 360-degree fulldome environments. Every tiny detail is critical to sustaining the illusion of a fully immersive world.

Using Kaiber AI for our film

Kaiber AI is useful and quality, but I have a serious doubt: I don’t know if it works with fulldome video. I haven’t had time to experiment with it yet, but it will possibly help smooth out the scenes so that the assets can be merged even better. It works through AI directly on the final video files.

Artificial intelligence and cinema: Reverie AI

In anticipation of the launch of Reverie AI , we are stimulated by its promises to create virtual worlds for Unreal Engine almost by “copying” a pre-existing picture. Its ability to generate scenery that faithfully mimics reality, combined with its potential in color correction of virtual scenes, opens doors to unlimited possibilities in visual storytelling. Reverie AI promises to be an excellent complement to our workflow, improving visual consistency and ensuring smooth and convincing transitions between the virtual and real worlds.

Move.ai, cheap and working mocap

Another “smart” tool we plan to use for our film will be Move.ai, specifically the inexpensive service Move One (on launch offer at $15 a month, then expected to cost $30). With a simple iPhone app, we will be able to create simple, ready-made custom animations without too many fixes or clean-ups. This reduces time and cost, allowing you not to be limited by the animations already available in services such as Mixamo , ActorCore or the MoCap Market by Noitom.

Here a very quick video test:

Clearly, they also have the more expensive professional service that allows up to 8 rooms to be used. But we do not count on using it for this first production unless there is a very real need.

Speech-to-speech for film dubbing… Is it possible?

In our production, innovation does not stop with the creation of the digital world; in fact, we also want to take advantage of the ongoing technological revolution in traditional aspects of filmmaking, such as dubbing.

The voice of Luna and the other 3D avatars, must be consistent (not to say identical) in all media, both in the virtual and real worlds. We face a complex challenge: maintaining the same voice across multiple platforms, including social media and especially real-time streaming, without tying ourselves to a single voice actor. This allows us to have flexibility in storytelling and to adapt to various formats without depending on the availability of a specific actor.

The idea is to replicate the voices of some real actors and associate them with avatars (Luna, Kyle, and China for starters), then transforming any actor’s voice, in real time, into Luna’s, to use it for both dubbing movies and social content (as much in the original language as in translations), and for the live streaming superimposing these entries on the animated avatar in motion capture.

From the excellent but expensive Respeecher, to the ambiguous Voicemod

We explored options such as Respeecher , an advanced speech-to-speech conversion tool, but the costs for real-time are prohibitive. We’re talking about 1,000 or 2,000 a month for a few hours of use. Voicemod presents itself as a cheaper solution because of its AI voices, although there are conflicting rumors about its reliability (some even consider it to be malware or cryptojacking…). And there remains the problem of not having the rights to the voice, which they own, which will certainly prove to be a problem in the future. I do not yet know the cost for real-time conversion of Resemble.ai , which I have used in the past for the much cheaper text-to-speech, and of Veritone Voice .

Another tool that I have not been able to test is Voidol 3 , at a cost of about $300. I couldn’t find a demo version, but I admit I didn’t even try that hard to request one. It is one of several Oriental software created because of the typical Japanese passion for the anime world. But actually adaptable to our purpose, as we will see in a moment with another Japanese.

MetaVoice Live, Mangio RVC Fork and W-Okada Voice Changer, free and open-source

The final solutions, after much research, are. MetaVoice Live and the Voice Changer from W-Okada. Both open source, which reassures us that we can base the “future life” of the characters on these services, and both free. I cannot fail to mention the YouTube channel AI Tools Search which has been most useful to me. Among the most interesting videos in this area, certainly this one:

I particularly like MetaVoice: it is under heated development and has a cloud version for non-real-time conversions that provides greater quality and flexibility. This one is not free, but the cost between $10 and $25 per month can all in all be dealt with.

In contrast, W-Okada ‘s Voice Changer has many independent developers developing solutions compatible with it, such as GitHub user Mangio621 who created the Mangio RVC Fork , a software with Web interface that can transform the voice of a real actor into that of the chosen voice model. Many of these models, especially famous people, are already available on sites such as Voice-Models.com in “.pth” format. A classic format used in machine learning), but the best thing is that with the same Mangio RVC we can perform training, or training, of a custom voice. All locally thanks to a good video card. And therefore clearly always available, and for free.

I will do a specific article on this shortly, however. So you will follow me as I do some interesting tests.

Generating video with artificial intelligence

One use of AI in film production may be text-to-video, or video-to-video. But how useful can generating videos with artificial intelligence be? I mean right from scratch, describing to the AI in text (or with a very simplified video reference) what you want to achieve. Somewhat the future, to date tools like Runway Gen-1 , Genmo AI , o Moonvalley AI are little more than experiments. Useful at some junctures, but far from the quality and realism needed for a film production.

Evidently, we will have to work for a few more years to get our films

Canon EOS R5C and Dual Fisheye lens for 3D fulldome video

What about live action filming? In our journey, we are trying our hand at a very exciting combination of equipment. We always keep the goal in mind: little expense, much return. I had originally planned to shoot everything in full 360 degrees, also to make it already compatible in case of future virtual reality porting. But the cost became prohibitive, both for the camera (a professional camera like the Insta 360 Titan , which by the way has not been updated for years, costs more than 17,000 euros), and for the difficulties of starting from the very beginning with such a wide field.

So the idea is to produce the right video for the dome, in 360×180 degrees (basically half the sphere). And the lens Canon RF 5.2mm F2.8 L Dual Fisheye , paired with the Canon EOS R5C , should prove to be a winning choice. This setup not only provides immersive images, but also allows us to experiment with stereoscopic shots, which add an extra level of depth and realism.

The Canon EOS R5C camera, with its dynamic range between 11 and 12 stops and 8K sensor, offers excellent value for money. This is an important consideration for us, as we are trying to maintain a balanced budget without sacrificing quality. We will also consider whether to rent it … From Adcom , lens and camera are offered at about 200 euros daily.

To be honest, we can’t fully exploit the potential of stereoscopy yet because of projection, but it is definitely something we would like to explore in the future. It is one of those things that looks really cool and could add a special touch to our project. Have you ever projected stereoscopic fulldomes or do you have any suggestions on how we could integrate it into our work? I would be happy to hear your thoughts and ideas.

And if it will be lacking in anything compared to more emblazoned rooms…. Again, we count on leveraging AI for video quality improvement. But research, in the field, still needs to be done.

Conclusions

In the end, our choice of digital tools and equipment reflects a desire not to compromise on quality while keeping an eye on the budget.

In short, we are creating something beyond traditional cinema. Thanks to artificial intelligence and cutting-edge technology, the “cinema of the future” is no longer a dream. It is real, we want to see it happen. And with the genius of Gérard Bernasconi to the screenplay (who also gave us a great technical contribution), the precision of Michela Sette in the role of VFX Supervisor and the creativity of Michele Pelosio as director, we are forging a revolutionary cinematic experience. Get ready, because we are about to take you to a world where cinema and reality merge into a transcendental experience

L'articolo AI in Film Production: How to Produce an Innovative Low-Budget Film è apparso primo per Future of Cinema - by Dario Riccio.

In his voice we can hear, without a doubt, a passion for the new way to watch movies, with an immersiveness beyond any level ever achieved. We could say that they have “recreated” the movies in the real world.

Let’s try to understand it a little better with himself. So here is the interview with Andrea Moccia.

Introducing Andrea Moccia

Dario:

Andrea Moccia is a veteran in innovative cinema who has been trying for years to innovate and change a bit the way cinema is enjoyed.

Andrea:

Hi Dario, nice to meet you. I am very glad to meet you.

Dario:

How are you? All right?

Andrea:

It’s all right, thank you. Here in London it is gray as hell. Today there is a fog I have never seen, it feels more like Switzerland than London.

Dario:

Okay, a Londoner telling me about the fog like that is interesting. People say it’s always there, but I’ve been to London a few times actually, and I’ve never caught it with fog….

Andrea:

Fog not too much. The sky is always gray. But fog not too much.

Dario:

Yes, indeed the sky was gray. I clearly remember that. So, tell us a little about you… You’re from Rome. So a pure Italian, from the capital. Moved to London since? What was it like? Why did you go to London? How did you find yourself there?

Andrea:

I moved to London in 2010 for studies. I graduated here, and during my last year of University, I was very lucky to be introduced to the world of Secret Cinema. I went to a couple of their shows, and among them there was one based on The Shawshank Redemption. After going to this show, I was absolutely impressed by this world, this format. And I found a way to start working with them.

At first as a trainee. While I used to study in the mornings for exams, I went to the office in the afternoons. Then I started working with them, first as an assistant and then for the last five or six years as a producer.

Dario:

Okay, I just stop you a moment, just to get a good understanding. First of all, I presume you were studying something related to cinema. What were you studying?

Andrea:

No, I did business administration and management. So not directly related to cinema. However, it is a degree with which I found many affinities as a producer. Let’s also say that I’ve always had an interest in art, in performance. When I was a kid I did some acting, I liked that world. And I found a way to apply my degree to the world of performance and art.

How Secret Cinema is interpreting immersive cinema

Dario:

OK. Because in fact Secret Cinema, even to explain it a little bit to our listeners, what is it? Very broad question.

Andrea:

Of course. So, Secret Cinema is a company that specializes in what we call “immersive cinema”. And what the company does, is basically take these big abandoned buildings, or big spaces, pick a movie and rebuild the whole set and world of the movie inside this building.

Then spectators buy a ticket. The moment they buy a ticket, they are assigned a character. We tell them how to dress and who they are within the world. Never anyone among the main characters. Always characters created by us who could exist in that world.

Dario:

Extras…

Andrea:

That’s right, extras basically. And people come to experience the world of this film. We have covered great classics such as Back to the Future, The Battle of Algiers, Star Wars. Up to much more niche films, for example Park Chan-wook‘s The Handmaiden, or I, Daniel Blake.

The company was founded in 2007. It operates on a global scale at this time. We have performances in the United States, England and China.

England, the ideal country to grow up

Dario:

Basically, however, you do most of the shows in London. Or they are somewhat evenly divided among countries.

Andrea:

Most of the shows we always did in London, yes. Let’s say London is our base, where we grew up. And England has been crucial to that growth, I think.

In terms of bureaucracy, but also as theatrical industry, there is already a base where something like this has been able to grow far more easily than in other countries. For example, in my opinion even if the idea had started in the USA, it would have been more complicated. Even just to access these buildings, which are usually large mansions not built to accommodate thousands of people for an event.

They can be abandoned factories, or warehouses, things like that. So the process of getting permits, the process of turning these buildings into what they are, then assets or spaces for very large events, is very complicated and expensive.

And I think, in the early days, these ingredients could have worked only in England.

Close cooperation with film companies

Dario:

Do you guys have any help, any cooperation from the film companies, or do they not care? Do you just buy the rights to reproduce the film?

Andrea:

Look, it really depends from movie to movie. There are some properties that are perhaps a bit more dormant, with titles that are not being redeveloped or, otherwise while having historical value, are not bringing in much revenue at this time.

At that point maybe the studios contact us and ask if we want to do something with these films; in short, they want to generate new interest. Then, of course, there are other films where we ask for and pay rights to the studios, working with them as licensees.

There is always collaboration at the operational level, though, never just at the financial level. There is always an involvement of “Studios” and creators in our creative process.

Dario:

Okay, so they are interested anyway. It is also a way for the companies to promote the film.

Andrea:

Absolutely. We make many films in years when a sequel is released. For example, Blade Runner, which we did in the year that Blade Runner 2049 was released. Or Star Wars in the year the franchise started again. So yes, many times there are also marketing interests behind it.

A beautiful thing Baz Luhrmann said to Fabien (Fabien Riggall, nda), the person who created Secret Cinema in 2007, after coming to a Moulin Rouge! show, was:

I made the movie, but you created the world.

Baz Luhrmann

And that, I think, is a very appropriate summary for what we do. We create a world of the film.

How to choose films suitable for immersiveness

Dario:

You are currently working with Marvel, and then with Guardians of the Galaxy. For example, who wanted to make this film that is now on the schedule? You asked Marvel or it was Marvel who asked you to develop the world, the marketing.

Andrea:

Guardians of the Galaxy is the last show that I worked on at Secret Cinema. I recently left Secret Cinema.

In reality, it is originated from a much larger collaboration between Disney and Secret Cinema. We have a contract of several years with Disney in which we have access to several of their titles. I mean, who wouldn’t want to work on the Marvel world? Who wouldn’t want to join the Marvel Universe…

Dario:

I guess.

Andrea:

Basically when we choose movies, we don’t always choose them because it’s a great movie or a great story. There are many beautiful films that do not work for a Secret Cinema, precisely because we have to create a world, we have to give viewers the opportunity to disguise themselves, to impersonate someone.

Guardians of the Galaxy, being all set in space with this mix of alien races, we felt it was the most interesting world we could recreate in a physical, real-world setting.

Whereas instead, I don’t know… One of my favorite Marvel movies is the first Iron Man. However, it does not offer many insights to engage viewers. While instead you come as an alien, to watch Guardians of the Galaxy. So, all the people come in makeup and in these beautiful, exaggerated costumes; and part of the beauty of coming to a Secret Cinema is just that, right?

There are people who spend weeks, months, creating costumes for themselves, putting all the photos of the process on Instagram. Many of our fans are also cosplayers, or people who work in the creative industry. It is also an opportunity for them to show off their creative talents in a social, sociable environment.

The typical structure of a Secret Cinema

Dario:

Oh yes, because the Secret Cinemas, practically, how are they structured? Is there the screening of the movie in costume, or you see the movie and then move on to other environments…

Andrea:

In a typical Secret Cinema, we sent you basically an email. The email redirects you to a website. On this site we usually have personality questions, questions that will get you assigned a character. Each character is part of larger groups, and each group has its own specific mission; its own specific path within the show.

Then, when you get to the show, the first two or three hours are like an open world video game, where you can go and do various missions; talk to all the characters. You go to sets, you can get drinks, you can go dancing, there’s music, there’s food… Yes, it’s like a video game where you choose your own adventure.

Dario:

A real video game, in short, rebuilt in reality…

Andrea:

Correct. So this is the beginning. Then, after two or three hours, you watch the movie. And during the film we also reenact many live scenes. So, for example, if you’re in Back to the Future and there’s the DeLorean leaving and going to 1986, there’s also a DeLorean going next to you. Just a real DeLorean passing by you.

Or, when Doc falls off the tower, we had a Doc starting from the clock tower on a zipline, going over the spectators and landing on the other side of the field. Because Back to the Future is a show we did outdoors.

For Star Wars, we had an X-Wing that flew over a thousand viewers, landed and out came Luke Skywalker. So it’s all things like that. These are shows that reach production values that few other events can achieve.

Is VR the future? Or will we get back together again?

Dario:

Of course, the parallel with virtual reality comes to mind. In the sense that you have similar experiences, with the major difference being that you are in a sociable environment, with others, and not alone locked in an headset. And this could also make a big difference.

In your opinion, or even according to people you have talked to (viewers, colleagues…), how will this situation evolve? Will people always want to be with each other?

Andrea:

I, at least, am an absolute fan of shared experiences. For me, there is nothing more beautiful than going to a concert with thousands of other people; or going to a movie or theater.

There is just a feeling that you can’t beat. Unless virtual reality will reach levels we cannot even imagine at this time. I don’t think it will come to replace the live experience.

What we are trying to do is to put some magic back into going to the cinema, right? Fabien, when he started this concept of Secret Cinema he was always thinking about the days when people went to the movies dressed up; when going to the movies was an event, when it was really a celebration.

This is what we want to report. It came into being almost a reaction to these multiplex giants. These multiscreens where the experience of going to the movies has been somewhat “formatted,” and the magic has been lost a bit. So this is what we are looking for. And this, in my opinion, is the challenge that cinema, and other art forms as well, will have to face.

But how do you do it? How do you put this magic back? How do you go about breaking a format that has come to produce this mechanization of entertainment? I don’t know how to explain…

So many seek a new way of conceiving cinema

Dario:

Yes, everything became very standardized. In fact, this is something I am glad to hear you say. Because I also support it, personally and in my blog. In mindset I am like that. I also mentioned you, in the email I sent you, that I follow a lot the studies and life of Douglas Trumbull, the creator, the maker of the special effects of 2001: A Space Odyssey.

Andrea:

But also of Blade Runner. I think he died recently, Douglas Trumbull.

Dario:

He died very recently, this year in February I think. He, however, has spent this life trying to find a solution. Obviously its was not Secret Cinema. It was something less “scenic”, but still perhaps more reproducible. That’s the difference.

Secret Cinema I think is a wonderful thing. I still don’t, I just met him recently, but I will definitely have to participate because it is beautiful as an idea. It is very expensive, though; organizing a single screening will cost impossible sums of money for a small business to handle.

Andrea:

Absolutely. However, there are manners. The shows they are doing now are shows of ten million and up per show. So very, very large productions, with very real economic needs. You have to sell hundreds of thousands of tickets to make these things work.

Skates and social issues: where Secret Cinema started.

But let’s remember where Secret Cinema started, though. Because Secret Cinema started with a screening in an abandoned skateboard park, where they basically projected that Paranoid Park on a sheet. And two hundred people came.

So there are simple ways to do things in an effective way, especially when you have films that maybe deal with serious and difficult issues. At that moment you don’t really want to go and recreate the film, but you want to bring the theme to light more. And so we also do a lot of social things.

For example, we did screenings against censorship when films were censored; then screenings in refugee camps. Cinema is not just entertainment, is it? There is a whole sector of cinema that is about entertainment, fun, and it is beautiful that it exists. But there is also a whole part of cinema that wants to send strong messages.

And that in my opinion also needs to be reinvented. Spaces must be found. I am a huge Marvel fan, but you can’t just go to see Marvel at the cinema. Other kinds of films should be seen as well, because there are beautiful things out there.

Dario:

Yes, and then cinema almost was born for that. Well, apart from the very first screenings, the development of cinema internationally was because of this. To pass messages, at the time perhaps more political than anything else, which then quickly became moral or various messages in the early decades.

Andrea:

But we may have forgotten that a little bit.

The importance of moral awareness in cinema

Dario:

Effectively, yes. Oh God, in part. From one point of view, even just many movies or series on Netflix still try to maintain a moral. Then, currently, even those have become standardized.

In the sense they go by fads. At this time I personally notice feminism or LGBT+ is trending; there is an attempt to raise awareness of these issues. Sometimes there is the topic of animals, meat, vegetarians. However, we often talk about fads that are then forgotten. When, instead, perhaps it should be more consistent.

Andrea:

Yes, it is true. And in my opinion, here again we come back to the issue of commerciality. If people don’t go to the cinema, don’t buy tickets, unfortunately these films will no longer be made. Ways have to be found to bring people back to the very act of going to the cinema.

I have been following Cinema America in Rome for many years, and what they have been doing: bringing cinema to the streets, and various initiatives just to rekindle the passion for film, to go to the cinema and watch something all together. Which I think after two years of pandemic we really have to “have it taught” almost, don’t we? People are perhaps still a little afraid to go out, to socialize and are definitely more cautious. However, there are ways to continue living without taking risks.

Dario:

In past years I have organized various outdoor cinemas around Italy. We have done cinemas in Tuscany, in Montalcino, in Ladispoli, and also in Rome, in various places around the capital, in Abruzzo… I mean, in different places and basically they were all very, very well attended.

Of course, they were free cinemas; they were almost always contributed by municipalities or associations. So there was no ticket clutch. But people liked them; they went there; they spent evenings together and they were all super happy to do something different.

The problem is that, precisely, they were happy to do something different. Instead, it should become the norm. I remember my father, born in 1937 (other times…) telling me about when he was a child, or at least a teenager, and the movie theater was kind of their bar. In the sense that they went to the movies almost every day. It cost very little, and they went there to be together and spend two hours. I mean, it was just different culture from this point of view, but then it evolved.

The death of the high street

Andrea:

And maybe that’s part of the solution too, right? The Internet, new technologies, have changed the way we do everything. For example, here in England we are experiencing a time when there is a lot of conversation around “the death of the high street“, the death of the street with all the stores. And how can we reinvent stores so they don’t die? How can we reinvent the high streets?

Stores are reinventing themselves. Here a clothing store, is now no longer just a clothing store. It is a clothing store with a cafe, with a bookstore. A bit of a gathering place. And perhaps cinemas also need to reinvent themselves and become gathering places. It would be great if, for example, all theaters had a small library on the history of movies…

I mean, in my opinion, you have to detach yourself a little bit from this format of cinema. You go, you get the popcorn, you get the Coke, you sit down, you just leave. If cinema is turned into an experience, if cinema is turned into a reason to go out, to socialize, in my opinion it will find its place again.

Dario:

Definitely. Experiences also and especially not reproducible at home. Because the problem with current cinema is that it is reproducible almost entirely at home, with a video projector or a simple TV. You can eat popcorns, you can drink Coca-Cola or beer. I mean, it lacks perhaps that extra reason that makes you say: I can only do this thing there. And we all are lazy, basically, as the human race.

Andrea (smiling):

Me first.

Dario:

Me too, don’t believe. And so, rightly you stay home. But yes, there should be one more reason.

Multi-sensory cinema: yes, but without distractions

Let’s talk about one last thing, then if you want to add more, otherwise we’ll end since we’ve already been half an hour together. I wanted to ask you, what do you think about using more than two senses at the cinema? Have you ever experimented with combining smells, possibly flavors, so a combination with foods or other things but always related to the film at the cinema? Also touch, albeit more complex (although in Secret Cinema there is actually touch, you can touch things of the film as well).

Andrea:

It is a trend that is going a lot, for example in Asian countries. There are some big companies that have started to make these really big 4D cinemas where they include “smell,” water splash, something like that. Or there are other cinemas, especially here, where for example during a screening they bring you food or drink.

So there are already things that unite them. For me, the question one must always ask himself is: how do these things go to enhance, to improve, the viewers’ experience? If these things are done with the intention that maybe a certain smell can improve my understanding of the film, of what I’m going to see, offering a new perspective, then I’m absolutely pro. If these things are just being done to do something new, then I think they can occasionally have the opposite effect. Becoming distractions rather than a magnifying glass.

Dario:

Okay, you say that they could become something that makes you lose the sense, the lesson. It goes back to the argument we were making earlier.

Andrea:

Correct. However, I am always “pro” experimenting, I am always “pro” trying new things. New things need to be tried, and they also go in this direction. Absolutely. However, as long as the intention is there. Now I can’t give you any concrete examples, but there must be a reason behind it. There must be an intention.

Dario:

Yes, I get the concept. Indeed an example is complex so, on the spot. It depends on the particular story, the single film, the single screening. It is a beautiful answer that makes total sense, though.

Because it is true that there are often so many things, or so many technologies, only done because you had to find a problem to the solution; not the solution to the problem. Which by the way, at the moment, is virtual reality: a totally useless technology the way it is structured that they are trying to find a use for.

Technology should help, not destroy

Andrea:

Yes, let’s say I am a bit, um, technophobic on these points. I really like analog. I’m one who read books, who have an agenda. In this respect I am a little behind.

There are some really cool virtual reality applications. There have been exhibitions here that I have gone to where it has been used really well. As there have been other really low-level things.

Dario:

In which a television set was fine.

Andrea:

That’s right, let’s say. In my opinion, the technology is still a few steps behind the ideas.

Dario:

It is true, exactly that. It goes a long way perhaps in gaming, because still we come back to the argument that spectacle, the visual of images, of the location, and the involvement do so much. But it’s the only area currently that’s really going… With education they’re trying to do something, they’re inventing, but we have to wait to really have value added to the traditional method that’s always been here, and maybe can be modernized with little.

Andrea:

Correct.

secretcinema.org, to learn about Secret Cinema

Dario:

Okay Andrea, I don’t know if you want to add anything else.

Andrea:

No Dario, thank you very much. It was a really interesting talk. Go to the Secret Cinema website (www.secretcinema.org, nda), and go see what it is. If you haven’t been, absolutely try to see it, because that are truly unique experiences that will certainly bring you closer to whatever movie you go to see at Secret Cinema.

Dario:

Okay. I also support the invitation; I have already gone to see the site and from that I want to move on to see the real Secret Cinema. So, I will have to go to London as soon as possible to do that. It’s really beautiful. An innovation like few in the world, and as such it deserves to continue.

Also to you Andrea good luck with everything. Directly for your career, your future, and any other innovative experiment you get involved in.

Andrea:

It’s going to be a very, very interesting year, I think. There are many interesting insights, so definitely maybe we’ll have a chat later on my new adventures.

Dario:

Definitely. That’s all I’m waiting for. Thank you, Andrea. Bye!

L'articolo Secret Cinema: immersive, but social è apparso primo per Future of Cinema - by Dario Riccio.

Articolo correlato sul mio blog: https://www.darioriccio.com/che-cosa-cinema-esperienziale/

L'articolo Che cos’è il cinema esperienziale è apparso primo per Future of Cinema - by Dario Riccio.

What is a media, or “mediated story”

Cinema belongs to the “media” category. Also definable as “mediated story”, or “telling a story through something”. They can be a microphone and a speaker (radio), a film camera and a projector (cinema), a camera and a television (TV), a word processor and a printing press (newspapers). Any way of telling a story that is not “direct”, therefore different from traditional conversations, is defined as media.

Among these, there are the new experiential media to offer users potentially positive and beneficial results, increasing their involvement and above all empathy. The experienced experience makes them feel active and involved in the story, seeing, hearing, touching or experiencing situations never experienced before.

The characteristics of traditional cinema

Each media has advantages and disadvantages. Or, we could say, possibility and impossibility. Going into details, let’s quickly see the five peculiar characteristics of the media at the center of our attention: cinema.

• Static and linear narrative structure
• Single or dual mode
• Episodic
• Third person perspective
• Passive audience

Static and linear narrative structure

It’s at the basis of cinema: a one-way narrative structure with beginning, development, climax and end. Fixed and static, it obviously cannot be changed by the viewer. Sure, there are Flashbacks or Flashforwards, but they don’t fundamentally change this approach to storytelling.

There is usually a causal chain: each passage narrated in the script leads to a subsequent passage which is a consequence of the previous one.

And the movies, of course, don’t change over time. Except for very rare cases of errors or problems occurred after the release of the film (for example Kubrick cut the last minutes of The Shining after its release in theaters), these remain the same from the moment of publication, and forever.

A good analysis was made in 1992 by Prof. David Pinault in Story-Telling Techniques in the Arabian Nights.

Non-linear structure in traditional cinema

In postmodern cinema there is actually a concrete attempt to modify this linear structure. Often attempts are made to “weave” the story, leaving the viewer to use intuition and irrational instinct to understand the film, while logic and drama are dissolved and diluted.

The linear structure often develops into a symphony composed of several parts with a non-linear structure. We will understand better in the following sections how this will develop even more in experiential / interactive cinema, forming various personalized narratives (which is reminiscent of videogames).

Single or dual mode

By this we mean that cinema involves a maximum of one or two senses, sight and hearing. Clearly the current cinema is always dual, as in the “single mode” category we can only insert silent films.

Over the years, various attempts have been made to improve the involvement of the other senses. We think of great directors who manage to convey the sense of taste to the viewer, for example in films such as Eat Drink Man Woman, Ratatouille or Mid-August Lunch. Clearly, it is a trick to our brains. But ultimately, cinema itself is.

Episodic

According to Prof. Jason Mittell in his book Complex TV: The Poetics of Contemporary Television Storytelling, current cinema and media are generally episodic; that is, they tend to develop around an event or a series of interrelated events. This is even more true, of course, in the world of journalistic media.

A story is told based on facts and events, real or fictional. It is told from the point of view of the narrator, which makes it easier for the viewer to enjoy but at the same time empathically distances him from the story.

Third person perspective

This perspective is an important feature in our analysis. In fact, although possible, in traditional cinema first and second person perspectives have always been little used. In the history of communication in general, it was abundantly used only in the radio of the first half of the 20th century.

Heart of Darkness, first-person radio adaptation

As an example, consider Joseph Conrad‘s Heart of Darkness. It was adapted for radio in 1938 by director Orson Welles (famous for having made Americans believe they were under Martian attack through the radio show War of the Worlds, in the same ’38). The goal was to have the protagonist tell the story in first person directly. Interesting how Welles himself, at the time new to cinema, tried to persuade RKO Pictures to make the film version.

It had what it takes to become one of the greatest films of all time and, perhaps, raise public awareness on issues that instead, misinterpreted, led to the Second World War the following year. But it was the use of the first person, in addition to the political themes little loved by the majors, that probably pushed Hollywood to not consider its feasibility. It was a drastic break from the rules, and the world wasn’t ready for it yet.

In fact, Coppola tried to recover in 1979 with Apocalypse Now, only freely inspired by the novel “Heart of Darkness” as it is set in Vietnam and not in Africa. It is certainly too late to assist peace in Europe and in the world.

First person in the history of cinema

There are some sporadic cases of filmic use of first-person narration, especially in its early years. Sometimes little known cases, but which somehow tried to change the way of seeing things. First of all I think of Dr. Jekyll and Mr. Hyde of 1931, better known in Italy as Dr. Jekyll and Mr. Hyde, by Rouben Mamoulian.

Other cases were William Dieterle‘s 1934 film The Firebird. Or even Lady in the Lake shot in 1947 by Robert Montgomery, and Dark Passage by David Goodis of the same year.

Two other examples, decidedly more modern, we will see shortly in the section on experiential cinema, as they are more useful for a comparison with the cinema to come.

I have cited only examples of works in first person, because talking about works in the third would be impossible or useless… They are practically almost all the films in existence. And then because, with a view to creating the cinema of the more experiential future, I believe that these ideas must be well taken into consideration.

Then you will tell me what you think, I’m interested in being closer to the technical world than to film criticism.

Passive audience

As we said, the story in the current or past media is told in the name and on behalf of the narrator. Whether it’s the writer or the director, the journalist or the speaker, everyone shows you what they want. Have you ever wanted to look at something to the side, or behind the camera, but the direction didn’t show it to you?

It happens particularly often with sport on TV, and it is perhaps for this reason that television itself is among the first media to have “progressed” towards a direction decided by the viewer. In Italy, as Wenner Gatta mentions, when the “Spidercam” was inserted on the football fields, the viewer could choose with the remote control whether to see the meeting with the classic direction or through the spider camera only. And from there, above all Sky has continued to develop technology a lot by exploiting multiple transmission channels for the same event.

After a study of the cinema that was, and still is, we are finally going to analyze the characteristics of the new cinema, projected towards the future, to also understand how to overcome the problems of the last 127 years.

The characteristics of experiential cinema: the future

The five characteristics that we will be able to find, all or in part, in the cinema of the future, according to what technology currently makes available, can be:

• Immersivity
• Interactivity, non-linearity and sociability
• Multi-sensory presentation
• Algorithmic, customized in real time thanks to data
• First person perspective

Immersive Cinema

We already know some partially experiential media. We think of immersive virtual reality and augmented reality platforms. As we saw in the last article, these are fully inserted in the line of continuity between real and virtual world hypothesized by Paul Milgram.

Immersion means “enveloping the user in a real physical space using augmented or mixed reality on a portable or wearable device, also including haptic interfaces“.

I found one of the first practical examples in the paper “Situated Documentaries: Embedding Multimedia Presentations in the Real World“, by Tobias Höllerer, Steven Feiner and John Pavlik and taken from the International Symposium on Wearable Computers of 1999.

These “set documentaries” were entirely based on wearables, to integrate novels and documentaries in real-world locations. The system is very reminiscent of the current AR glasses, obviously with the technology available in 1999. It was a backpack with GPS tracker and 360 ° video camera (developed by Shree Nayar, of Columbia University), a type of handheld computer with graphics, audio and video, and augmented reality glasses capable of signaling in the real environment the points of interest. It was called MJW (Mobile Journalist Workstation).

The glasses could also reproduce rudimentary 360 ° videos superimposed on the real, and the gaze was the main aiming system. By staring at an object in the real world for at least half a second, it was selected producing related info and multimedia files. You could also travel in time by touching the desired year on the handheld display.

A great example of what augmented reality will be 20 years later. AR is in fact developing rapidly, becoming in common use since 2018 following the implementation in iPhones of the native ARKit api (which, by the way, I believe to date is the latest noteworthy innovation in the smartphone field).

The first point of our cinema will therefore be immersion. But certainly a different immersion from that seen so far. The basis will in fact be the story told, the cinema. We will not immerse ourselves in the world, we will not immerse ourselves alone or in virtual common environments. The immersion will be produced by the 360 ° screen, by the stereoscopy, and by the “hall”, or dome, with inside elements directly linked to the narrated story.

The writers will have to be really good, to develop plots that make the viewer feel “involved”, without being taken for granted. As is obvious, the first thing that jumps to my mind: to keep the protagonist of the story constantly seated, who must impersonate us spectators. It is at best a starting point that should not be underestimated, but above all the brain storming will be really interesting in the early days.

Interactive, non-linear and social

Word to the viewer; choice. These may be the keywords for the non-linear structure of experiential cinema; which is therefore more complex.

Although without particular rules, it maintains its own logic based on temporal sequence (or order), duration and frequency, which are the concept of division proposed by the French essayist Gérard Gennette in the field of literary fiction, then introduced in the field of film criticism by Andre Gaudreault and David Bordwell.

The order of events in the stories

In this section we open a general parenthesis on the stories; there are in fact characteristics in common with the future immersive cinema. Gennette clarified that the event could have occurred:

• before the narration (analyses or flashbacks)

• in the future (therefore only announced or expected, the prolixes).

• Again, events can be narrated in a different order from how they happened (anachronia), used to make the story more compelling.

• More rarely there can be a movement between one narrative level and another (metalepsis).

An example is the “author’s metalexy”, a sort of passage by the author from external to internal to the story, or conversely if a character becomes a narrator.

Another literary example is of the poet Virgil, who “kills” Dido in canto IV of the Aeneid, or Diderot who writes in Jacques the Fatalist: “Who could prevent me from marrying the Master and making him a beak?”. Both examples taken from Armando Mollica Bonivento‘s doctoral thesis at the Ca’Foscari University (in Italian), which I invite you to read for greater understanding.

The duration of events in the stories

So, after the order (event that happened before, future event, narration in order different from the real or passage between different narration levels), we have the duration. Also definable as the “rhythm”, the “speed” at which events are told.

We can mainly divide it into four types:

• ellipses (very accelerated rhythm), with frequent chronological jumps;
• synthesis (relatively fast pace), in which a story is summarized in its main points. They can be of variable length.
• scene: relatively slow, it is the classic narration almost in real time. An example are the dialogues;
• descriptive: no progress in history, we stop to describe a certain moment.

Clearly, these types can be combined. We can have, for example, a synthesis inserted within a dialogue.

The frequency of events in the stories

Frequency is nothing more than the relationship between how many times a certain event occurs in reality (even invented), and how many it is told. If, in practice, the same event is narrated several times (or the same statement of a character repeated).

I leave you the link to an interesting article on the issue, from which the following image is taken.

The cinema of the future has an intersubjective non-linear structure

After this long interlude, let’s try to understand why the story, or the screenplay, is decidedly more complex for the cinema of the future. In fact, all these elements must be inserted within an intersubjective non-linear structure. That is, the story can go back and forth in time, in an environment common to other viewers who may want to make different choices from ours.

Intersubjectivity is the biggest problem to be solved in writing new scripts. In fact, if interactivity is a concept already well known thanks to video games, interactivity in common between several people, with necessarily a screen that shows everyone the same images, presumes that a small democracy is created inside the room.

The intersubjective non-linear structure has a great advantage: it respects the viewer. It grants the right to choose one’s own story, to judge the morality of some scenes. The viewer becomes the center of the film as well as a part of it. The current cinema is definitely too one-sided, and if it has lived unchanged for so many years it is only for the simplicity (inherent in the characteristics) of using it as a means of political and commercial propaganda.

Thinking about the average use of cinema, which is also and above all a moment of relaxation and not very active entertainment, we must not however fall into the temptation to insert excessive “gamification”, transforming it into a video game. I mean, nowadays we go to the cinema to relax with friends or family, to spend time without thinking too much. And choosing implies thinking … This is why interactivity must be limited and not even mandatory, and it probably won’t be the top priority in creating the cinema of the future.

We will see in some future articles why interactive cinema has not been successful in, albeit few, past experiences. But it is related to this.

Multi-sensory presentation

Experiential media may seem like something new in recent years, but it’s not entirely true. For centuries, humanity has developed its peculiar characteristics, improving the technology available in small but constant steps.

Wearable devices to engage the senses

According to what was reported by the Prof. John V. Pavlik in the book “Journalism in the Age of Virtual Reality“, the first experience of” wearables “devices can be summarized in the Chinese invention of the ring / abacus, wearable measuring instrument dating back to the Qing dynasty of the seventeenth century.

Following in Europe, in 1780 the “pedometer” was developed, a step counter, to arrive in 1965 with the American attempt (unsuccessful) to create the first exoskeleton (Hardiman) to allow humans to lift up to 650 kg.

In recent years, developments have certainly been much faster, also thanks to the logarithmic scale of the technology that hardly stops once it has started. To understand, did you know that in 2004, just 18 years ago, the GoPro wearable camera came out … And that it even used 35mm film?

Cinema stimulates us physiologically and sensually

My body is not just an object among all objects, but an object sensitive to all others, which reverberates at all sounds, vibrates in all colors and gives words their primal meaning through the way it receives them.

Maurice Merleau-Ponty in Phenomenology of Perception

At the beginning of this paragraph I inserted the trailer for the film Piano Lessons, Jane Campion‘s 1993 masterpiece. I chose it as an excellent example of how current cinema tries, in more or less orthodox ways, to deceive the brain in order to involve senses not directly involved (touch in this case). I also invite you to review the last article which talked about Matthew Shifrin and his Legos for the blind.

This is the magic of cinema. Art in this sector has reached unimaginable heights, even if you try to think about how to go further. How to materially stimulate the other senses. Although already in the 1940s, the philosopher Siegfried Kracauer wrote:

The material elements that appear in the films directly stimulate the material layers of the human being: his nerves, his senses, his entire physiological substance.

Siegfried Kracauer

How to involve the five senses in the cinema

How then to involve the five senses, or at least more than two, in cinema? It will be necessary to proceed in stages, as technology allows it. So let’s consider the experiments done in the past, to later understand how WE can engage the five senses for our viewers.

Touch

Meanwhile, the touch: William Castle, in 1959 shot the horror The Tingler. He inserted a vibrating device called “Percepto!” Into the seats of some cinemas, which was synchronized with the action. Castle himself was an evil genius … Before the screening of Macabre in 1958, he had everyone deliver a $ 1,000 insurance policy in the event of death or fear during the film. Then during the 1959 film House on Haunted Hill, he let a phosphorescent skeleton enter above the stalls. Based on a pulley system called “Emergo“.

And finally there is “Illusion-O“, launched with the film 13 Ghosts: all the elements in the frame, with the exception of the ghosts, were subjected to a blue filter. The ghosts instead had a red filter, and were superimposed on the frame. The audience received cards with red and blue filters: looking through the blue filter, you couldn’t see the ghosts. Through the red filter, however, they could be seen.

Still on the touch, obviously the modern 4D cinemas that we all know have the entire vibrating chairs, as well as ropes that touch the legs, fans for the wind effect (hot or cold) and sprinkles of water for humidity.

Smell

Smell: a strategy was adopted by John Waters for the 1981 film Polyester. This is the “Olorama” system, basically cards with scratch numbers. Each number has a smell (rose, pizza etc …), which can be smelled at the required moment (a small number appeared on the screen).

Another 1960 attempt was Smell-O-Vision, used only in Scent of Mystery produced by Mike Todd Jr (son of the famous Mike Todd Senior, author of Around the World in 80 Days). Smell-O-Vision involved the introduction of up to 30 evocative smells into the stalls through tubes that led to individual seats in the room, with perfume bottles held on a rotating drum.

Today a company has taken up the concept in a modern industrial production, called Olorama. With whom it would be nice to collaborate; objectively, the system seems much more functional than smelling cards (then actually reused only in a couple of children’s films).

The criticisms he received are interesting. According to the Times, some viewers complained of delays between the smell and the scene, others found the scents mixed in an unpleasant way, Henny Youngman said he didn’t understand the film because he had a cold.

Other inconveniences to pay close attention to are nausea and headaches caused by too strong and persistent fragrances, possible discomfort, distractions and heavy air.

For the sake of completeness of information, I quote Walter Reade Jr‘s AromaRama. Key difference with Smell-O-Vision? Simply, the AromaRama used the air conditioning system for the diffusion of aromas. Cunning.

Eating with another is a way of saying: “I’m with you, I like you, let’s form a community together”.

Thomas C. Foster

Taste

We have seen touch and smell, taste is obviously lacking. Although progress has been made towards systems of “transmission” of taste (through objects to be licked), I do not think they are yet developed enough and, above all, we are not yet ready to welcome them. And I don’t know if we ever will …

However, during some screenings of Willy Wonka & the Chocolate Factory, Wonka chocolates were provided to the spectators. And from this I had the idea of directly providing the screened food, really to the spectators. A double advantage: for the identification in the film, and for the cinema economy and customers who would respectively sell and buy better food than popcorn and Coca-Cola.

I soon discovered how this idea is not exactly original: in 2012 in London, more precisely in Notting Hill, Edible Cinema took place. It was a collaboration between the Soho House team, organizer Polly Betton and experimental food designer Andrew Stellitano. Basically, each present had numbered bags and glasses containing food and drink, on the armchairs there was also a menu explaining the meals, and a woman appeared on the side of the screen during the film to indicate the time to eat or drink each number.

Algorithmic, customized in real time thanks to data

In a data-centric society, cinema cannot stand by and watch. Of course, in respect of privacy and possibly without using this data for not very noble purposes.

A great use could come from the geolocation inside the dome, to eventually send different signals to the different spectators. But, above all, it will be possible to take into account the direction of the gaze of the latter to understand what is more interesting, and to have a more passive “input device”, and therefore less tiring, due to the interactivity we have just talked about.

Other anonymized data, such as physiological responses moment by moment, may be useful for the development of subsequent films and evaluate the reactions of the audience. Which, in a subsequent development of experiential cinema, can also be exploited within the same story (for example, interactively manage volumes to increase or decrease human reactions).

Women’s Aid, a prime example of algorithmic interactive advertising

At World Women’s Day in 2015, an interactive blow-up depicting the face of a woman victim of violence was installed in the Canary Wharf business center in London. A face detection camera was used to update a counter and change the image whenever a passerby paid attention to the advertisement. It was a typical example, albeit in the advanced advertising sector, of using data to modify the result obtained.

Artificial intelligence for storytelling

Already today many stories are created with artificial intelligence, as for example the American Associated Press does. The Times also created an algorithmic robot, the QuakeBot, which automatically acquires data from the U.S. Geological Survey, the US organization of earthquake analysis, to automatically write the complete article with magnitude, epicenter and time. The human editor only has to verify its correctness and publish.

These robots, these artificial intelligences, are led to the development of increasingly engaging, interactive and multisensory stories. What will be fundamental for the cinema of the future, to assist human screenwriters in the writing of increasingly complex and autonomously unmanageable stories.

Furthermore, they will be able to exploit the data present in the world, and in the cinemas themselves. To create “tailor-made” stories for the target audience.

Artificial intelligence itself is, and will increasingly be, used in technical video production. Google, for example, has the Jump compiler (well described in their own paper), which deals with the union, stitching, of 16 high quality video streams to obtain a complete 360 ° VR video. The main result obtained was the elimination of much of the latency, to therefore increase the sense of reality of the image shown.

Ultimately, AI will certainly be the center of attention. And, to avoid controversy, it must also be used with lead gloves, focusing on privacy and the importance of the human being.

First person perspective

We already know some partially experiential media. We think of immersive virtual reality and augmented reality platforms. As we saw in the last article, these are fully inserted in the line of continuity between real and virtual world hypothesized by Paul Milgram.

These are clearly in the first person, as the real protagonist of the story is ourselves. Experience is given by contact (although still virtual), and by direct observation of objects and events in the ways we like best.

Experiential cinema will often have to be in the first person. This is unlike most of the present and past films, which instead aim to tell us stories with eyes outside the story. But ours will be a different first person: we will be the spectators, the main character. Recently, in 2016, the film Hardcore, directed by the Russian musician Ilya Najšuller, had a good success. Evidently producing cinema from a cultural background outside of it helps to take risks and innovate it.

The whole Hardcore is shot in first person. I believe the success is well deserved, and it is a good point of reference for the cinema to come. A way of narrating video games, which invites us into someone else’s life, looking at the world through his eyes. Well, I think the only difference is that it will have to be our life, inserted in the new cinema. I know, it’s scary, but the writers shouldn’t tell us that either …

I also want to mention another film, not entirely in first person (the protagonist can be seen in various scenes, removing the effect of total identification) but which has a decidedly more constructed and engaging plot than Hardcore: Enter the Void, from 2009, directed by Argentine director Gaspar Noé.

Two films that I recommend you see, first because it will be a special and different experience. Second, to get a taste (albeit reduced) of what cinema will be like in a few years.

To identify everyone in the history of experiential cinema

How can you build a film for many, which can realistically represent the life of each of them? Once again, the writers will have to do a great job of multiple introspection. By creating engaging but generalist stories, inserting characters who will also be new to the protagonist in the story. No known relatives, perhaps a distant cousin we didn’t know we had. Nothing more.

In 2015, on the occasion of the fourth international conference on Design, User Experience and Usability in Los Angeles, Aaron Marcus collected in a very useful book a series of information on the current relationship between computer science, the virtual world and human beings. Based on the assumption that each of us has our own culture and knowledge, the design (and therefore who creates it) must leave us the freedom to experiment, to have doubts and thoughts, it must build a world accepted by all the personal cultures of the spectators and, at the same time, propose his own idea of the universe.

We want and need to reflect, to understand. And this absurdly leads to the hyperrealism, that is, to a construction of the virtual world that is very faithful to the real world, in which to experience and be present in the first person.

The importance of sociality in the cinema of the future

We have seen, speaking of the first-person perspective, that technology now allows us to decide a place, at a given historical moment, and live virtually in it. But one thing will be important: we will have to live it with others.

Sociality is a theme that should not be underestimated: there is no life alone. To reconstruct the real in the virtual, other human beings (and living beings in general) must therefore necessarily be present. And a concrete interaction with them.

Basically, reality is an inherent concept in our mind. Many things are real to you and me, while others are only real to one of us. The “mediated” reality has an inherent complexity in this, mainly because it must be able to reproduce a synthesis of our personal realities.

Let me explain better with an example: Aurora has a habit of calling her mother in case of problems. In her reality, her mother is always present. He gives her advice, embraces her, offers her constant support. Marco, on the other hand, has a bad relationship with his mother. She has always had problems, she has tried for years to help her but to no avail.

Aurora’s reality requires a loving and very present mother, on the contrary that of Marco rejects her. The writers of the new cinema will therefore have to be able to fall into hyperrealism, without causing moral incidents with any of the spectators. How is this resolved? Important characters may, or perhaps should, be ambiguous. An ambiguous character that allows everyone to “accept” it by mixing their own inner reality with that reproduced and therefore “mediated”. In short, see them as you wish.

Ultimately, the “static” part of the real world, such as trees and houses, is always there. And easily reproducible. On the other hand, the human part, the social part, represents a definitely more complex choice, even if not insuperable.

L'articolo What is experiential cinema è apparso primo per Future of Cinema - by Dario Riccio.

Current cinema, a modern derivation of the ancient analog, has five characteristics: static and linear structure, dual mode, episodic, mainly in third person and not interactive. We will see later what each of these voices means, while I continue to think about what the cinema of the future will be, about immersive cinema, and how to make a good prototype.

I started this article on a Swiss train, and finished it at Barcelona’s charming Espai Joliu. I love trains, being able to work or relax while moving between two locations. Among my corporate benefits is the general season ticket for all means of transport in Switzerland: my luck. A few days off from work, I take the opportunity to reach Domodossola (the first Italian town across the border), or wander the length and breadth of the country of the Helvetians, which reserves surprises and magical corners almost everywhere.

How important is immersion in the cinema of the future

I am part of the line of thought, now increasingly in vogue, which imagines a single evolution for cinema: immersive cinema. According to purists, or “traditionalists”, cinema must be left as it is. But we have already seen that, after all, they are not necessarily right.

As you may have noticed, I am not at all cadenced in publishing articles on my blog. I don’t like online editorial plans, I prefer quality over quantity, avoiding in-depth articles only to come out “Friday at 7”. This article took me days of research, maybe more, and clearly blogging is not my primary activity, but it has resulted in a roundup of ideas and characters that I hope will be a definitive answer to the question: how will the cinema of the future be?

Few have seriously tried to innovate it, and in this article we will get to know some of them. Certainly not for lack of a market. Rather for lack of resources, lately more and more projected to the lucrative, and more manageable, virtual world.

This article follows from: Innovating modern cinema

Advantages and disadvantages of modern cinema

I reflected on the advantages of cinema as it is conceived today:

• It can be used on the move, even here on the train.
• Other actions can be taken, delegating the mere role of companion to the audiovisual.
• It can be used in company.

I ask you for other advantages, I really want you to comment. Turning to the disadvantages:

• The methods of use have been updated little for decades.
• It is not able to “fill” all the senses of the viewer.

So let’s create a system, using already existing technology, able to minimize these disadvantages.

Analysis of the advantages and disadvantages of modern cinema

1) You can watch cinema on the go

Clearly, “old style” cinema by definition is destined for theaters. However, in order to recover costs, a production must be usable by as many people as possible. A good substitute for the small screen are virtual reality viewers, which allow you to view stereoscopic 360 ° videos.

2) You can do other things while watching a movie

Here virtual reality, at present, shows all its limits due to the complete isolation it creates. Bypassing this limit is perhaps possible with augmented reality, or with a serious upgrade of existing virtual reality technology.

3) You can watch a movie in company

By designing an update of the cinema system intended as a “cinema”, for decades there has been a technology used above all in scientific dissemination: the fulldome, ie projections in a dome. However, the same system is not new to film experimentation, even by great masters of the past.

4) The methods of use have been updated little for decades

The first disadvantage of cinema is rather an observation. If, on the one hand, great directors or companies have brought immense innovations in shooting methods (think ofAvatar di Cameron con una Motion Capture all’ennesima potenza, o alla Industrial Light & Magic che insieme ad Epic ha dato il là ad una seria Virtual Production), poco si è fatto per le sale.

The transition from film to digital, without wanting to diminish it, is comparable to the change of our home TV. Best quality, lower distribution costs, maximum stereoscopy but otherwise everything as before.

5) It is not able to “fill” all the senses of the viewer.

A century ago, a great innovator, today’s cinema fails to offer that “something more” compared to other more modern tools.

When looking at an iPad or modern TV, the field of view covered by the image is approximately 25°. The cinema screen, on average, covers a field of view of 50° on the 360 of the sphere (it depends a lot on the position in the room). It is necessary to increase it, if you really want to return to offer that “wow” effect, due to the immersion, which is worth the trip and the ticket.

The acoustics are already very good; Dolby Digital systems allow excellent immersion. What is missing is still the feel, the smell and, why not, also the taste.

Why virtual reality is not popular

I’ve been trying to figure out why you never want to put a helmet on, and a recent University of Glasgow paper / survey by Laura Bajorunaite, Stephen Brewster and Julie R. Williamsoncertainly helped.

The paper deals specifically with the use of VR viewers in public transport, which in itself is a place of frequent cinematic use. But the discourse is expandable, with the necessary adaptations, a little more generally.

At the base there are clearly reasons of personal safety, but also of comfort. Social acceptance is also still a long way off, making VR users “stupid” in the eyes of other passengers. Progress is still being made to make one’s virtual world more integrated with reality.

For example, occasionally throwing an eye to the real world with the cameras installed on the viewer, or having indications on the positioning of other human beings in real space, can reassure us. A complete Virtual Reality must be avoided in favor of Augmented Virtuality; as already defined in 1999 by Prof. Paul Milgram in its line of continuity from real world to virtual world.

I invite you to read the paper to learn more, but the concept we must take into account is that total immersion is a problem.

Immersive cinema

We have already covered the history of fulldome cinema in detail (article available here), but To the Moon and Beyond, is considered by many to be the first immersive film in history. The first to cover a 360 ° field of view, filmed with the Cinerama technique using a single fish-eye lens, on 70mm film at 18 fps, and projected onto the roof of a large dome about 30 meters high.

If its use was still somehow “didactic”, we moved away from the astronomical canons to bring instead a greater involvement towards the issues of transport. The film also influenced Stanley Kubrick, hiring its creators as VFX consultants for the legendary 2001: A Space Odyssey. Lester Novros and our next protagonist: Douglas Trumbull.

Douglas Trumbull, a mission: to save cinema

Douglas Trumbull was a man with a great idea:movies can be more realistic and more immersive than they already are.

I would have liked to meet him in person, he was a myth for me. I tried to write to him a couple of times, offering to visit him at his Trumbull Studios in the countryside of Massachusetts, but unfortunately I got no reply. Perhaps the addresses I had were no longer current, perhaps he was too busy as well as old. I don’t know, the fact is that he is no longer with us except through his studies that still have the time to change the history of cinema.

He was one of the greatest experts in the world of special effects, first analog and then digital. Creator of amazing graphic masterpieces in the top of last century Hollywood production (2001: A Space Odyssey, Close Encounters of the Third Kind, Star Trek …). After starting to smell the decay of traditional cinemas, he devoted his entire life to finding a solution. Which he too found in immersive cinema, as a true pioneer.

Perhaps due to his analogue habit of always finding physical and real ideas rather than creating a new app or a new system to monetize users’ time while they are alone or at home, Douglas was a multinational craftsman. He certainly had art in his DNA: his father, Donald (Pappy Trumbull to friends) had been involved with VFX since the 1930s. Not officially accredited, he worked on The Wizard of Oz from 1939.

It was during the filming of 2001: A Space Odyssey that, only 25, Doug created the legendary psychedelic color stream “Slit Scan” based on the technique developed shortly before by John Whitney for Vertigo.

Immersivity is the future of cinema

I report part of an interview made by Filmmakermagazine.com to Trumbull himself.

If you look at the industry, it is moving more and more towards immersive experiences. Take virtual reality, for example; the problem is that people don’t want to wear the headset. Immersiveness is what TV cannot offer; when 3D took off, everyone started making 3D TVs. But now nobody makes them anymore.

I think the problem was miniaturization. When you look at a 3D image on the big screen, you have a 6 foot (1.82 meters) figure standing in front of you, while when you put it on a TV screen you have like a 6 inch (15 cm) figure. When I took the projects we were working on and played them on the TV screen to see what they looked like, the effect was completely lost.

Douglas Trumbull

Magi Pod: innovating the cinema

Douglas Trumbull in his later years developed with Trumbull Studios a system that united many of the key points for the renewal of cinemas. A redesign of both shooting and projection systems. At its core, 120fps 4K projection solves a problem Kubrick already denounced over 50 years ago: 24fps blur is obnoxious and unrealistic. For this already 2001: A Space Odyssey was shot at a high framerate, although almost no room was equipped for reproduction.

Trumbull wanted to bring back the magic he himself felt in visiting the curved screens of the Cinerama in childhood. But technically there were many problems to be solved.

Troubleshoot 3D projection

To begin with, stereoscopic projection requires halving the framerate (or doubling the projectors, which would be uneconomical). This means 60 fps projection per eye.

Then there is another, more concrete problem: in 3D films both images (right and left) are recorded simultaneously. But they are then projected one after the other, alternately (right, left, right, left eye etc …). There is therefore a small difference that is not visually perceptible, but not negligible from our unconscious. Solution? Record the two images with a distance of 180 degrees / shutter, then alternately, as they will then be projected. This solves a second problem, which concerns the “cinematic look”.

The high framerate is not in fact used in cinema because it makes the product too similar to video and TV. Trumbull then concluded to increase the flicker effect originally given by the cinematic 24 fps thanks to the recording of the left and right frames of the stereoscopy with 180 degrees / shutter distance. In addition to reducing the luminance of the screen (in the Magi equal to 14 FtL, less than 50 lux), but not reducing the gain of the screen itself.

The entire Magi patent aims to increase audience involvement: it translates into projection in the Magi Pod.

A small cinema, 70 seats ll in the best viewing area; toroidal silver screen (curved both horizontally and vertically) with 3x gain to increase contrast and therefore immersion; 32 speakers for enhanced surround effect; rotary subwoofers capable of reaching 1 Hertz, creating a “4D” effect with the absurd vibrations.

Trumbull’s years of experience leading the technical departments of the biggest blockbusters, in a single, magical, immersive cinema. If knowledge must always go forward, it cannot be ignored in the design of the cinema of the future.

Cross reflections in fulldome cinema

By designing a fulldome cinema, the differences with the Magi Pod are many and fundamental. The Magi does not in fact fill a dome, but only a part of it. And you cannot simply “enlarge” the field of view to compose 360 degrees. There is a first big problem: cross reflexes.

Anyone who has worked with high gain projection screens will have realized that a silver, 3X amplified screen is a kind of mirror. Reflecting both the light coming from the outside and the same image projected from one side to the other in the case of a convex screen. A problem that also experienced Steven Spielberg’s 1991 production, Back to the Future: The Ride; fulldome re-proposition of the original Back to the Future.

By limiting the radius, and keeping the spectators within an optimal viewing angle (therefore in central positions) the problem is reduced to almost zero. But a complete sphere must necessarily be a white screen with a reflection ratio of less than 1, or the solution is still to be studied. And in this case we will certainly have to do practical experiments in the near future.

Immersion in the cinema of the future

In fact, almost every previous study we talk about in this article is in some way linked to the digital world. But, as you may know, I am a staunch supporter of sociability and the return to real life.

The room will have to resemble a space and time machine, where you will enter a totally new dimension for the duration of the screening, and from which you will come out with a complete experience. Above all, in company.

The world is heading towards the conquest of space, which will initially be within the possibilities of a select few. Others will want to try new experiences, and they will have to do so while necessarily being on the ground.

Five senses for immersive cinema

The viewer will have to be the protagonist, so the narrative will also have to change. We remember well: without disturbing the intuition, the sixth, we still have five other senses.

Today’s dual cinema manages to cover only two of them: sight and hearing. And, especially for sight, we can do a lot to improve the situation. Reading the number of Wired Italia of December 2021 (magazine that is always full of interesting ideas) I found this paragraph in an article by Henry Jenkins (lecturer at University of Southern California, director of the comparative media studies program at MIT in Boston for more than 10 years; not the last of the arrivals):

I am convinced that the metaverse will not be the future both because devices such as Oculus viewers are terrible (especially for those who, like me, wear glasses), and because it is not the visual experience that guarantees the highest level of immersion, but that audio and haptics, as evidenced by decades of studies and pioneers such as Metthew Shifrin and Nonny de la Peña.

Henry Jenkins

Jenkins’s idea, similar to mine in many ways, intrigued me to get to know musician Matthew Shifrin.

Matthew Shifrin: Legos and the cinema of the future

Matthew Shifrin is a young musician, blind from birth. Since he was a child, a Lego enthusiast, these allow him to experiment, touch, “see” without eyes. Thanks to touch, and therefore also to Legos, he can understand how the world works; as did. According to his eloquent example, he will never know what the Statue of Liberty is like without climbing it. And climbing the Statue of Liberty can be problematic… Therefore, a small Lego statue perfectly fulfills its function.

It is his thirteenth year of life. At the door rings Lilya, family friend. She approaches Matthew with light steps, looks at him and says cheerfully: “I have something for you!”. Matthew, confused, smiles at her as the woman places a package in her hands. The child opens it, touching a sheet entirely written in braille: these are the instructions for building Lego structures by himself.. His eyes shine, he is so happy. And he wants to get to work right away.

Lego for the blind

The story leads him to found, years later, “Lego for the blind“. An association, with its website, where you can download instructions like these for about thirty Lego products. And convinces the same company to include them in new products. He also created a system to help the blind to climb the rocks, really brilliant.

Definitely an interesting character; but you may be wondering why we talk about him in an article on the cinema of the future. I understand you, I was so taken by his story that for a moment I did it too. Then I remembered where I wanted to go.

In his TEDx Talk at Suffolk University, starting at minute six, he explains it well and in a very funny way.

Blind people in the cinema of the future

The point is: if the world manages to be immersive for a blind person, cinema can never be immersive in a similar or equal way to the real world if it is not for a blind person. The fact that the cinema of the future can be enjoyed in a better way also by people with disabilities, can only be an excellent and stimulating side effect.

While studying singing and accordion at the New England Conservatory in Boston, he continues to look for ways to make modern life more suitable for people like him.

In episode 3 of his Blind Guy Travels podcast, Matthew describes going to the movies with his friend Ben. The need to have Ben to describe the visual part of every single scene, and even thus not have a complete idea of the whole, leads him to create with his university a tactile device for the cinema.

With the support of the New England Conservatory’s Entrepreneurial Musicianship Department, Matt and his business partner have developed a programmable vibrating vest integrated with technology that mimics the sensations of overturning, flying and falling. This allows a new audience to effectively follow the Marvel Daredevil saga, a blind lawyer who becomes a superhero at night.

A few days ago I wrote an email to the conservatory asking for more details on this system, unfortunately without receiving an answer. I plan to comment on this article if I have any news in the future, it is interesting.

We will talk about experiential cinema

I have ready an analysis of the characteristics of current cinema, and of how they evolve in experiential cinema. How the latter is interactive, immersive, algorithmic and other important studies for the purpose of creating our cinema of the future.

But I decided to end the article here, it was getting too long and dispersed. I have already copied the text into a new WordPress post, which I will complete as soon as possible and publish.

I renew the invitation to comment, to share doubts and criticisms, and above all to participate in this innovation if you like it. You can also write to me, my address is dario.riccio@gmail.com. See you soon!

L'articolo Immersive cinema is the future è apparso primo per Future of Cinema - by Dario Riccio.

A more democratic cinema

Virtual Production (I also had to write the entry on Wikipedia Italy ), motion capture, internet, digital and above all low-cost training, allow you to no longer be blocked by heavy budgets. Even new directors, or technicians, can have their say with a sustainable model. In the past, low-budget often meant super-low-gain. Without the support of public funding (which especially in our latitudes bring forward only Chippuò‘s friends and relatives), it was impossible to think of producing a cinematographic work, even a medium-level documentary, without losing thousands of euros.

All this reflected on the costs to the public, dictated by the large lobbies that made it inconvenient for many poor families to enjoy the Seventh Art. A day at the cinema for a family still has a cost comparable to months of Netflix. The benefits of digital distribution are enormous, but this isn’t meant to be the only method.

Personal skills are more important than budget

A great one-man-band, today he can potentially create a movie on his own. And above all, a good quality film. The participation of influencers can even help with distribution (I’ll do an article on this soon, now that I think about it). It was unimaginable until a few years ago, right?

Of course, it has always been possible to make no-budget cinemas. Thousands, perhaps millions, of young directors have done so in the past. But we were talking about very different results: without rooms worth tens or hundreds of thousands of dollars (Red et al.), Without international studies, without huge teams of specialized technicians (which were rare and expensive, given the inherent difficulties in training), he was at the maximum possible to create films that are not on the market and of a quality tending more to the low level than to the medium. Technical quality, but also limitations in storytelling caused by the technical impossibility of reproducing many ingenious ideas.

Today, this is no longer the case. A well-armored computer, with a top graphics card and a real-time render engine (among which the most popular in the cinema field, Unreal Engine, is distributed free of charge …) allows you to obtain 3D graphics that have little to envy to the big . And they can make small producers grow a lot in the cinema of the future.

Technologies for the cinema of the future

Let’s move on to analyze some of the technologies that could help us in the new cinematic use. Initially I expected to insert more than one already in this article, but given the length of time I preferred to limit myself to talking here about the fulldome video, promising myself an article in the coming days that will continue the discussion.

Fulldome video

Fulldome video projection is a technology that I have supported for years. It is the visualization of a video projection on the roof of a dome, to have a hemispherical overview. One of the best ways to feel “inside” a story, without using (and introverting) virtual reality viewers.

Brief history of planetariums

The technology is far from new. It is in fact used for scientific dissemination in planetariums all over the world. The oldest dome-shaped construction known today dates back to 500 BC, built by the Etruscans. The history of art, and the visit to churches and ancient structures in every corner of the world, teaches us how this form has often been used in various eras to represent the sky, Paradise and the firmament.

To know the first “working” planetarium (I mean, in motion), we had to wait until 1229 when Frederick II of Swabia, Emperor of the Holy Roman Empire, he brought back to Italy a tent with a domed roof on which a mechanical planetarium was hung: a clockwork mechanism, with a globe in the center (representing the Sun), from which some arms that support the planets then known depart. To learn more about the history of mechanical planetariums, this book from 1965 is interesting (the reading is free and open).

The main disadvantage, or difference, of the first planetariums was the depiction of the sky in reverse: the observer saw the solar system as if it were outside it, and not on the Earth. This was still the case in 1774 when the astronomer Eise Eisinga built, by himself, what is today the oldest planetarium still in operation. You can visit it if you pass through the city of Franeker, in the Netherlands.

Between the nineteenth and twentieth centuries, the innovations were many and rapid. The birth of Carl Zeiss in 1846 drastically increased the quality of optical instruments (which before then were produced without preliminary design, let’s say “by trial and error”).

We then moved on to ever more precise mechanical planetariums, and the Atwood Globe in Chicago made history. A metal dome of about 5 meters built in 1913, with 692 holes to represent the stars, and a lamp to represent the Sun. For the first time, the concept of the planetarium was the modern one: the viewer saw the sky, as if it were on Earth under the real celestial vault.

The first modern planetariums

During the twentieth century, the evolution continued, the planetariums went from rare instruments available to kings and scientists, to places of scientific dissemination open to all.

The first projection planetarium was built by Zeiss, which in 1923 tested the first Carl Zeiss planetary projector in its 16-meter dome. Even today, obviously updated, they are used by some large planetariums as an alternative to digital systems. In the following years, planetariums opened in Munich, Rome, Moscow, Stockholm, Milan, Hamburg and the first non-European: the Chicago planetarium. This went on all over the world until 1983, the year of the digital turning point. The turning point, which led me to talk about planetariums in an article on the cinema of the future.

How the fulldome video was born

Here we are at a fundamental point: how the fulldome video was born.

1983, we said. We are in the States, in Richmond. More specifically, at the Virginia Science Museum. Here the world sees a digital video projector in a planetarium for the first time. It was different from the projectors we are used to today: the Digistar I, manufactured by Evans & Sutherland, used calligraphic projection. I admit to being ignorant on the subject, never in my life have I had anything to do with this technology. And the short Wikipedia entry isn’t much helpful. Of course, it was a vector (therefore, by definition, unrealistic and poorly detailed) and monochromatic (created by a single laser beam) projection. Can anyone help us understand the principle of operation in the comments? It would be interesting, out of pure historical curiosity.

The fact is, then the 90s arrived with the first DLP and LCD projectors. In 1996, Japan’s GoTo Inc. gave birth to the Virtuarium, specifically designed for projection of stereoscopic scientific videos in a planetary dome. Two years later, Sky-Scan introduced SkyVision, the first fulldome digital animation.

The equipment was still prohibitively expensive, but we were close to what is now a digital fulldome planetarium. And that it could become the cinema of the future.

Cheap fulldome cinema: is it possible?

The problem with digital planetariums was the cost of the equipment. A high resolution and good contrast video projector in the 90s and 2000s had very high costs, tens or hundreds of thousands of euros. Even more so with a good fisheye lens able to project on a hemispherical surface without noticeable distortions. To solve the problem we must therefore go back a little, indeed a lot … On October 20, 1939, when New Yorker James S. Conant submitted to the patent office what would become three years later the patent US2299682A.

Fulldome projection with convex mirror

The patent dealt with a shooting and projection system (photographic, for the time) that we could define as “Newtonian”, from the name of the inventor of the reflection telescope. The technique was not used in planetariums until 2003, when Paul Bourke, an Australian university professor, retook it to adapt it to the then growing business of small digital planetariums.

The principle is very simple: the image is projected through a classic lens for flat screens (a normal video projector, so to speak) towards a convex mirror very similar to the surveillance mirrors that until a few years ago could be seen in the corners of every supermarket. This mirror then deflects the image towards the dome ceiling, allowing it to maintain the right proportions.

To maintain good details, it is important to avoid secondary reflections due to ghosting as much as possible. This is why we usually avoid cheap surveillance mirrors, preferring (better to say, necessitating) first-surface mirrors. They are mirrors with a reflective surface (silver, usually aluminum or silver), superimposed on a support. The opposite of what happens in normal mirrors (called second surface) which have the reflective material behind another transparent such as glass or acrylic (to protect it). For more information, the mirror manufacturer Abrisa has written an interesting article on the differences between the two types of mirrors.

The problem with these mirrors is the delicacy, and the risk of blackening of the same (which must therefore be treated periodically). For the purposes of a fulldome projection, silvering with aluminum, less expensive, would also be good, since the system used for Newtonian telescopes has scientific needs of a very different nature (to maximize the frequencies of the reflected light spectrum).

Doubts of the scientific community

The Planetarian magazine, edited by the International Planetarium Society, in the March 2001 issue published various comments about the probable use, in the future, of planetariums as a place of entertainment unrelated to scientific purposes. Interesting Ken Miller, when he mentions the owner of a small cinema who, after trying various technological innovations, claimed that the cause of a bad income is always a bad movie, a bad script and a bad acting. The message counts, not the medium. And in this I agree in part. I can argue from my experience that the same message by a better medium remains a better message.

Will the cinema of the future be fulldome?

In this article it is still too early to give a definitive answer. In the coming weeks we will analyze other details and other technologies, but I remain fairly convinced that the fulldome can be a great compromise in the 21st century. Technology has accustomed us to increasingly immersive tools, and investments in virtual reality and augmented reality in recent years are pushing in this direction.

I was very excited about virtual reality, but lately I’m starting to think that its future may be less rosy than expected. I await Apple‘s entry into the sector to get a clearer idea, for now the main sponsor is Meta Inc., a.k.a. Facebook, a company that despite common opinion has never stood out for its ability to predict future. A company that has grown thanks to a single good idea of its founder Zuckemberg and his ability to attract investments, but which has subsequently developed only thanks to the acquisition of good, almost alternative ideas that work well in the same sector.

Virtual reality will still have a good success for the purposes for which it was initially designed. It has strong potential in gaming and in cultural and educational contexts, and its media push may serve to increase interest in an innovation of storytelling in images to get closer to a “collective virtual reality”.

As I imagine the situation to date, it will be augmented reality that will take us the most time during the day. An analysis on the use of AR for interactivity in the cinema of the future will also be fundamental.

As usual, I would be very happy if we could create a debate. I therefore invite you to comment, criticize or give new ideas under this article. And thanking you, I give you an appointment in the next few days for a next trip.

L'articolo Fulldome video: the cinema of the future? è apparso primo per Future of Cinema - by Dario Riccio.

I come from an interactive exhibition on Deep Fake in the arts, hosted in the pavilions of the Polytechnic University of Lausanne (EPFL). This Swiss institution, which in 2019 developed the technology for the recognition of “human” deepfakes, is a world-class excellence in the field of science and technology. He wants to help us understand the future of technology in art.

The theme is very topical; digital allows you to create artistic fakes that are almost indistinguishable from the original. But it also allows the use of the works away from their places of origin, to circumvent political or security problems. And it is on this that the organizers have concentrated, to evaluate the ability of a copy to cause lasting emotions over time.

The tools of technology in art

In a modern exhibition, interactivity is everything. For years I have supported the “museum”, intended as large rooms / parking of objects, dead in most cases. Exceptions are some masterpieces that still today attract thousands if not millions of experts and onlookers for their fame, but this is not the norm. In Switzerland, over 75% of museums do not reach 5,000 visitors a year. But even in Italy, attendance has increased in recent years (excluding the pandemic period). This suggests that we are moving in the right direction to modernize the service offered.

The tools used to add technology to the world of art are mainly touch screens, holograms, mixed, augmented and virtual reality.

Multi-view, or multi-viewer 3D projection

Mea culpa, I was not aware of a beautiful technology: multi-view, or multi-viewer, 3D projection. Operation is simple on paper, albeit technically complex due to the huge amount of data to be processed. A video projector, specifically the Insight 4K HFR 360 (toy worth over $ 300,000), projects 360 frames per second in 4K resolution. This exceptional framerate, combined with active 3D glasses (obviously synchronized with that framerate), allows for the simultaneous reproduction of 3 different stereoscopic videos, giving 60 fps to each single eye.

I tried it and, apart from the inherent discomfort in wearing active glasses for a user of “normal” glasses like me, the vision is perfect.

To further improve the situation, 6DOF tracking (with six degrees of freedom, understands where we are looking and where we are in the room). System similar, if not the same, to those used for the HTC Vive VR headsets (or the old Oculus). In simple terms, the 3D video changes perspective as we move around the room, to achieve the effect of “seeing behind” the projected objects. A change of parallax, copied from virtual reality. And here too, a nice way to add technology to art.

A note on the protagonist: the digitization with laser and scanner techniques of the Abbey of Monte San Michele in Bamburg (Germany). Closed in 2012 to be restored following major structural damage, it can therefore be virtually visited.

Automatic 3D photogrammetry with CultArm3D

In an increasingly automated world, where humans are needed less and less (does this seem like a problem to you?), It is no longer impressive to see a robotic arm photographing a mask far and wide to recreate it in 3D. It’s theCultArm3D.

It is specifically a 3D reproduction of the Samurai mask from the Edo period. And this is the world’s first scanner capable of producing a perfect 3D copy using automatic 3D photogrammetry. Produced by the research of the Fraunhofer Institute for Computer Graphics, of Darmstadt (Germany), it is able to calculate the size of the object, to scan it (even in the case of reflective materials), to recreate it in real time in color in 3D and then to be used in virtual or augmented reality, or printed.

Realistic 3D printing is possible thanks to the development of adaptive color 3D printing. In this specific case, the technologies of the Israeli Stratasys are used. All with a declared goal: to show ancient works without the risk of ruining the originals.

As I see it, well but not very well… Poetry is missing in such an object, and I don’t know how many would substitute it for the true original. The fact remains that in cases other than printing, such a machine allows significant time savings for the digitization of world cultural heritage (and in Italy, there is a lot to do).

Artificial intelligence puts us on the alert

This is what is first imagined when thinking about the future of technology in art. Nice, and at the same time a bit terrifying, Trust AI invites us to sit in front of her, take off the mask so that she can see better and answer her questions.

Her face is a hologram, she informs us of our personal data (age, sex, hair color) and progressively pirates our face until it assumes our appearance. In her speech, after having “stolen” her personality, she is keen to alert us to the data we share online; how these can be used for less than noble purposes by thieves.

Does artificial intelligence cause dangers? Certainly yes, but no more than many other technological innovations. And art installations like this help the public to understand how it works, in order to better defend itself.

The work was produced by Bernd Lintermann and Florian Hertweck at the ZKM | Hertz-Lab, in response to the disturbing spread of deepfakes. I invite you to read the long article in which they describe the project; it’s definitely interesting.

Augmented reality to visit historical monuments

Beautiful, fun for children, but apart from that, visiting a cave reproduced in 3D, displayed in augmented reality with an iPad in hand, I don’t think is the best way to use technology in art. I find it useless, a pure exercise in style; I do not want the creators who have indeed been exceptional.

Mind you, we are talking about one of the Mogao caves, near the city of Dunhaung (China) dating back to the Tang dynasty. Therefore, something very ancient endangered by the excess of tourism. Digital representations are welcome, but this is 100% the field of virtual reality: a trespassing of AR does not produce the same results.

Holding the big tablet in your hand is uncomfortable, to begin with. And since it’s all reconstructed in 3D, why not take advantage of VR to feel like you’re really there, and not walk into a black cube with white textures drawn on the wall?

Augmented reality can really do a lot to improve the quality of a tourist visit, but in the real place. You can get help from an iPad to reproduce life scenes from the past, legends, descriptive videos of a particular point… But a mere virtual visit, let’s do it right.

Augmented reality that makes us think

As soon as you enter the exhibition, two works arouse curiosity. The “Reclining Pan” which we will talk about shortly, and an isolated white pedestal. This is the basis for “The Golden Calf” by Jeffrey Shaw. A calf that takes shape only by pointing the object with an iPad which, thanks to augmented reality, reproduces it.

The importance of The Golden Calf is very clear thinking about the year of production: 1994. Shaw was a true pioneer in the world of augmented reality, and obviously in its original form there was not an iPad but a screen equipped with motion tracking. magnetic Polhemus, one of the very few companies to offer this technology at the end of the last century.

The most interesting effect are the reflections on the calf. Four cameras placed on the sides of the pedestal reproduce our image on the subject to give it that extra touch of realism. And to make the work always different, to want to make the viewer himself a co-author. And the calf, elusive, is the object of our desires.

3D printing for the protection of the sculpture

The “Reclining Pan” is the modern copy of an ancient copy. The original work is attributed to the Florentine sculptor Francesco Da Sangallo, built around 1535. In turn, it was a copy of which, however, the original has been lost.

The contemporary artist Oliver Laric made a 3D scan of it (which you can download for free from Three D Scans), then printing it in many different materials. The result is a new, modern, and provocative work. It calls into question the veracity of the post-original in the modern era, rejecting the postulate of the singularity of art and its private property.

Laric, for his popular work, has made many works scanned in 3D available, free of charge, to anyone who wants to download them from the Three D Scans website.

New paths for the future of technology in art

I have not described here all the works, which were over 20. I wanted to talk about those that most of all make us reflect on the changes that technology is bringing to the world of art. In addition to these, it is also worth mentioning the blockchain which here was represented by the multimedia work “89 seconds to Alcázar“, by Eve Sussman. Videos whose rights have been divided into 2,304 “atoms” of 20 × 20 pixels, and sold via blockchain. And whose EPFL reproduction was full of holes, as some atom owners did not consent to free public viewing.

A bright future, in which the public domain will be increasingly important. But at the same time, a future in which artists will have more and more ways to experience their creations, to spread them to the whole world.

In which it will be easier to organize thematic art exhibitions, even in small realities, without the enormous costs of transporting works of great value.

In which spectators will be able to admire and study works of art from all over the world, and from all eras, without the need to travel thousands of kilometers. Which, for heaven’s sake, is beautiful. But sometimes, and for some, impossible.

L'articolo The future of technology in art è apparso primo per Future of Cinema - by Dario Riccio.

On a break from work in the Swiss mountains, under the gray sky of early February, I lose myself thinking about modern cinema. What he sees in empty halls (and no, Covid is only partially involved). The one that sees series work more than feature films. Hollywood, like Bollywood or our Cinecittà, bet everything on this. But can modern cinema really go on like this? Or, at least, do we really want to lose a place for fun, socialization, so important as the cinema was in the last century?

A brief history of modern cinema

When Rossellini was at the “Modern Cinema”

I remember fondly when my father told me about the “Patini” cinema in Castel di Sangro, our hometown, in the days of his childhood. Born in 1937, immediately after the World War II the situation in Europe was anything but rosy. There was so much to rebuild, we lived in a climate of constant tension due to the Cold War.

The United States was the only power in shape, and mass cinema could only come from there. True, we were in the McCarthyism with not very clear rules. This name comes from Senator Joseph McCarthy, who persecuted and imprisoned hundreds of progressives in 1953/54. In a climate of collective political hysteria, every innovative idea was associated with the Communists. Characters such as actor Charlie Chaplin, directors Jules Dassin and Joseph Losey were forced into exile. The rebels, out of all the “Hollywood Ten” (10 screenwriters who appealed to American freedom of expression), rightly ended up in the cage. Walt Disney and others escaped by denouncing colleagues… If you can dream it, surely you can (which then, this phrase is from Imagineer Disney Tom Fitzgerald, not Walt as is commonly thought).

The cinema of the United States to conquer the world

Despite this, America came out with its head held high. The history of cinema teaches us that the major film companies joined forces under the MPEAA (Motion Picture Export Association of America) to subdue opponents in the international market, and the government of the country led by Roosevelt first and Truman then, took advantage of its hegemonic power to circumvent foreign protectionist laws.

But was it all easy? In fact, no. In any case, the public had to be convinced, it was necessary to give them serious reasons to reach the theaters TV grew in importance every year, and for this reason the focus was on technological and stylistic innovation. Among these, color was a cinematic exclusive. This Italian article by Domenico Palattella is interesting, from which I quote:

Color must in fact be understood as the natural evolution of black and white and of Cinema as art, because if Cinema is the art of reality, color also had to necessarily rise to enhance the emotions that only this Art can offer in a way so penetrating, yesterday as today.

Domenico Palattella

Color films increased from 20 to 50% of productions in the early 1950s, becoming almost all in the 1960s. The Kodak Eastman Color film (and, before that, Ferraniacolor in Italy), which cut costs, came to the rescue for its cheapness; but it was only a consequence of what the market was asking for at the time.

Another innovation was the change of format. The 4:3 TV ratio gave way to the more eye-catching panoramic format. Then again, young people were drawn to the new drive-ins. There was a real competition, TV vs Cinema. The baby boom convinced filmmakers to look for every valid way to seduce young people, obviously also by producing TV series: they have always been popular. But the cinema, intended as the theater, was not left to itself.

Attendance at Italian cinemas throughout history

I was saying, my father… He was dying to go to the cinema, and Italy was not America. There was very little money, and 100 lire for a child was really a lot. So he saved himself, and yes… Sometimes you could access the Patini Cinema for free by making eye contact with the owner. But what a desire to go to the cinema!

Is this still the case today? How is the attendance rate of cinemas around the world evolving? For Italy, Istat comes to the rescue. A 2018 report photographs the situation from 1993 to 2017. In this period, the percentage of people who went to the cinema more than 7 times in a year has dropped by almost 20%. On the contrary, the percentage of those who went at least once, has risen. Maybe, only to remember if the place hasn’t changed a bit. This increase was helped by the increase in the female audience (which, strangely, previously seemed less fascinated by the big screen).

Other important data? More and more over-65s go to theaters: from 6.3% in 1993 to 15.2% in 2017. Positive or alarming figure? I’m honest, I can’t convince myself of the answer. To close this quick overview (if you’re curious, read the report), the main reason why you don’t go to the cinema is (coincidentally): “I prefer to watch movies at home”.

Modern cinema costs too much

And unfortunately it is boring now.

But let’s go back a little. Let’s enter our virtual time machine, and let’s go into the first half of the 1900s. Even in Italy, in 1940 on average each inhabitant went to the cinema 8 times a year. In 1950 thirteen, to reach a peak of about 17 times in 1955. From there, a steady decline. 10 admissions per year in 1970, 4 in 1980 to stabilize at around two or less in the last 30 years. Causes? TV (and to follow the internet), but also ticket prices constantly increasing until 1990. And, I might add, lack of real innovation in an increasingly innovative world. Don’t get me wrong: in production there are, and many. It is in the fruition that innovations are lacking, and perhaps it is even more important.

Cinemas in the United States from 1930 to today

A look beyond the ocean? The US market is certainly more authoritative than the Italian one, not least because of the numbers. But it is no different… At the beginning of the 1930s (about 35 years after the opening of the first movie theater, in 1894), over 70% of the population went to the cinema at least once a week. 70%! This percentage dropped to around 40% in 1935, to then exceed 60% again in 1944. Since then, a continuous loss of spectators: in 1950 it was 30%, in 1960 less than twenty. And it was the 1964 the year that, in the US market, marks the stabilization of regular cinema-goers at about 10% of the population.

A clear reason, even in the States, was the price of the ticket. Hollywood has always been an industry whose rulers celebrate themselves as superior to the “plebs”, no matter what you think. And in lean times, they would never have given up part of their millionaire earnings. If there were fewer spectators, the price goes up. Simple. Too bad it’s a dog biting its own tail.

Movie theaters create sociality and sharing

Money was not the only reason, however: there was also a social reason (which returns cyclically): during the Great Depression, in the 1930s, cinema was a way to escape the cold, heat and loneliness. Movie theaters brought together strangers, who one next to the other shared the only social moment available to all.

3D movies, the only real innovation in recent years

I remember the period when stereoscopic cinema, “3D films”, made a comeback the last time. Not many years ago, I was in the middle of my film academy (2011, good times). Here, it could be a little breath of freshness. But the idea was rejected, officially because it didn’t work. However, between the need to accustom the public and the time necessary to update theaters and productions, the real reason for not reaching results was (once again) the will of the cinema world to speculate on customers.

Stereoscopy, on balance, was not a way to offer users more. It was a way to dramatically increase the cost of the ticket. And I myself, as much as I loved seeing some films in 3D, often preferred the 2D version just to avoid the 12/13 euro entry. How many like me?

The final solution: unique and economical experience

So, is there a solution? In my opinion, yes, and there are those who support me with more authority. Well, I get off the pedestal: I support him. I will tell you about it in the next few days, my break is over, but it is basically a question of returning to making cinema a unique, unrepeatable, fun, social and, last but not least, budget experience.

Do we want to reform cinema? Let’s make a revolution together? Great things always start from the bottom… And we are in the period when it is easier to do so.

Article F.A.Q.

L'articolo Innovating modern cinema è apparso primo per Future of Cinema - by Dario Riccio.