For the last year I have been grabbing time between lectures to make some progress on part 2 of the documentary talk about music hall. I have also been finishing some storyboarding for a couple of proposed films and some preparation for a BBC project, so it has been a full year! (That is by way of a preamble and an excuse for tardiness!)
Here is the full documentation on a piece I have just finished animating which is based on a song by Harry Champion:
Turn on the television anytime and the chances are you are going hear music that is produced using digital orchestration and sample libraries. So, the orchestra you are hearing is not, in the conventional sense, an orchestra at all – it just sounds like one. If the music production is done well, it should be almost impossible to distinguish between what you are hearing and what a “real” orchestra would sound like. That is the art / science of digital orchestration. The music for CSI, The Mentalist, Dexter and many more, is all produced “inside the box”. It is now almost impossible to find an advert where the music is not produced in this way. Tight production schedules and limited budgets usually don’t allow for recordings of real orchestras, so the sample based recording isn’t just a demo or a mock-up, it is the final cue.
Click on the link below. Here, there are two examples of an orchestral piece by Vaughan Williams and two examples of a Beethoven string quartet. One of each piece is produced using digital orchestration – which is which?
More and more, and for the same reasons, this also applies to films. Low and mid-range budget films will never be able to finance studio recording sessions with a large orchestra, music director, producer, recording and mixing engineers etc. Even in some blockbuster films where large Hollywood style orchestras are used, the soundtrack will often be enhanced with extra sampled percussion and brass etc.
What is sampling and how does it work?
Not Robots and Nothing to do with Synthesizers!
When you hear sampled music, you are hearing real musicians playing real instruments, – a solo violinist may well be playing a Stradivarius! – but, every single note and articulation has been sampled – recorded – separately and often many times over. Every note an instrument can play and in every way – ppp, pp, p, mp, mf, f, ff, fff, sfz, staccato, detache, legato, flautando, harmonic, con sordini, pizz, and so on, is sampled in meticulous detail. Often, but not always, these samples will be recorded “dry”. That is, with absolutely no ambient sound, echo, reverb. If you listen to dry samples, they are very uninspiring – if you have ever been in an anechoic chamber you will get the idea. Reverb to taste is added later. More about reverb anon. The world leading Vienna Symphonic Library (VSL) have built their “Silent Stage” so that all their sampled recordings are completely dry.
A full orchestral sample library will go through this process for every instrument. In addition, where sections of players are required e.g. First Violins, Second Violins etc. there may well be 14 or more players playing together. Full professional sample libraries are massive containing millions of samples.
The Legato Revolution
An important milestone in orchestral sampling occurred in 2002 with the development by Vienna Symphonic Library of the “interval legato concept”. Anyone who has learned to play an instrument to a reasonable standard will be aware that it is not enough just to play the correct notes in the correct order in the correct time. Good phrasing, and especially legato phrasing, will utterly transform a piece. Hitherto all sample libraries faced the same problem – how to join notes together so the phrase or passage sounded completely authentic. In 2002 VSL developed complex algorithms which overcame this problem, the results were astounding and this process has been taken up by all sample developers ever since.
Click on the link below to listen to a 1st Violin section from VSL.
Around the same time VSL also developed the method of using different samples for repeated notes. Most music contains many parts where the same note is repeated many times. If the same sample is repeated many times the result is not at all convincing and is known as the “machine gun effect”. The developers created a system where each repeated note triggered a different sample of the same note giving a much more realistic sound – this is now universally referred to as “round robins” and all serious sound libraries use this system.
Software
What do you do with 10 million samples?
Fortunately the companies producing such vast libraries also produce highly sophisticated software to control and manipulate the samples. The screenshots below show part of the user interfaces for VSL’s Vienna Instruments and East West Quantum Leap’s PLAY software. It is here that the user controls and manipulates the samples. There are many other companies with similar products but these are the two with which I am most familiar and use on a daily basis.
Usually there is a matrix where the user loads the samples required. In some pieces only a few articulations will be required e.g. sustain and legato, for other more complex pieces a wide range of individual articulations will be needed. This can be very demanding on computing power and often slave computers with powerful processors. a lot of RAM and several solid state hard drives, will be needed to take the strain.
A set-up like this can be used as stand-alone connected to a keyboard where the player can play, for example, an incredibly realistic string or brass section. This is commonly used in “live” situations. This of course has been and continues to be controversial as many musicians see it as a threat to their jobs and there is no doubt that this does and will continue to happen. On Broadway in 2003 there was a Musicians Strike which began as a strike over pit minimums and rapidly escalated. In London’s West End Cameron Mackintosh moved Les Miserables from the Palace Theatre to the Queen’s Theatre leading to a reduced orchestral size – a smaller pit – and increased use of samples triggered from keyboards. So, from the audience’s perspective there was no reduction in sound produced by the orchestra.
In music production for TV, Films and CD type work these interfaces will normally connect to a DAW (Digital Audio Workstation) such as Cubase, Protools, Studio One etc. Here the user controls the whole audio project with as many instrument tracks as required – it could be over a hundred.
Below are examples of editing widows from Cubase.
The coloured track lanes are individual instruments which are processed as required and eventually mixed down to stereo, or surround sound. It is in these widows that all the minutiae of editing takes place. The process here is similar to standard multi-track recording which of course is the staple of many recording studios. In some cases this is still done using multi-track tape recorders – analogue – but more and more, using computers – digital. Each method has its own champions and many a heated discussion takes place over their relative merits.
How Pleasant To Know Mr Lear
Screenshot 3 is from “Following Lear” Zontul Films Ltd currently in production. The music was composed and produced by David Watson (Kanon Digital Orchestration) using the digital orchestration techniques described above. A mixed stereo file was sent through the ether to Shonk Studios Oxford where soprano Thomasin Trezise recorded the vocals which were then sent back to Kanon for mixing and mastering. This track is not finalised but you can listen to the work-in-progress by clicking on the link below. You can also see Tim Wilson preparing some of the storyboards and animations for the piece.
A Note on Reverb
I mentioned above that many sample libraries are recorded completely dry i.e. with no ambient sound on the recording. This has the benefit of giving the producer much greater control of the final audio track. Reverb techniques have come on a long way since the days of the old “spring” reverb. This was literally a spring inside a plastic pipe which could be inserted into the audio path. It gave pretty impressive results for its time but if you want to see one now you will have to go to a museum!
There is a vast array of very high quality hardware and software reverb systems available. Where pure acoustic realism is sought, rather than a reverb or echo “effect”, the most commonly used system is that of Convolution Reverb or Impulse Response. Without going into the physics of the process, convolution reverb involves audio engineers going to a particular venue – a concert hall, scoring studio, a church or even a famous cathedral and actually recording the directional sound impulses of that space. Using these programs and impulses you can then place your “dry” instrumentalists or singers “virtually” in that space. The realism of the results can be quite amazing.
VSL’s Multi Impulse Response convolution (MIR)
VSL’s MIR, which I use a lot, goes even further taking up to 5000 impulse responses per room. They will also consider the placement of any instrument on a stage and the directional and sonic characteristics of the instrument. Microphone placement is very important and it is possible to listen as the conductor, someone sitting near the front of the auditorium or at the back or in the balcony and so on.
Mozart wet and dry.
Click on the link below to listen to excerpts from the Overture to The Marriage of Figaro.
This first one I have kept mercifully short. It sounds raucous, out of balance and truly awful. Yet, apart from the addition of the MIR impulses and instrument placement, it is identical to the second example which I hope is more pleasing to the ear. This second example is placed in the Mozartsaal in Vienna.
Where to now?
The speed of technological advance is unlikely to slow down. Computers are getting more powerful, processors ever faster, solid state hard drives will become the norm – this can reduce the need for huge amounts of RAM. The professional sample library market is very competitive with new top-end products advertised in specialist magazines every month. The digital / virtual orchestration area of the music business is almost unrecognisable from that of just a decade ago.
Where does that leave our living breathing musicians? The business will continue to change and musicians will have to adapt, there is no doubt about that. However, music conservatories across the world continue to produce incredibly talented musicians. Perhaps the British – but not the Scottish – Government has come up with a solution – stop producing musicians! The introduction of the Ebac. The Ebac will essentially be compulsory for all maintained schools. The Ebac does not include music and the arts! Any school which does not conform cannot be graded as outstanding. I feel sorry for the head teachers – especially those with a creative arts background – who will have to make very difficult decisions. I suppose we get the governments we deserve. After all we elect them. Or at least at the last election the 66% of us who voted did.
Perhaps there is something to be said for a benign dictatorship – especially if musicians are in charge!
Here is an 8 second bit of animation. He still needs the tail coat and details, but the basic movement is now beginning to take shape. I have abandoned the 4th finger like Disney… It really does take up time to animate a 4th!!
Here are some pictures showing progress on a scene we are finishing for the Edward lear film. This shows the Fatih gate which is that part of the wall that first gave way to the Turkish assault of 1453. Next to the fountains here is a museum with an astonishing diorama showing the actual bombardment of the city by the Turks. Here is a copy of the picture that Edward Lear painted. This is unusual among the Istanbul collection because it is in colour, so probably worked up by Lear a few years later.
Here is the completed picture we have just finished.
and here is the companion piece showing the walls a little further down.
There is a man I would like to meet. His name is Dan Lever. I saw him on the TV this morning and he was almost unable to speak because of the blustering nonsense of his adversary, a rather nasty piece of work called Simon Renton. Of course, Renton may be a perfectly agreeable man but he he was there to explain why University teachers were coming in for so much criticism. Instead, he tried to savage the present Government’s spending. Rather ridiculous as the rot really set in with the last Government and there has been a long steady decline in investment. A cannot imagine that Renton is a Conservative after this rant, but does he not realise that his socialist comrades look pretty feeble too? Dan Lever, in contrast, seems to be doing something about the problem and good on him! He has started a site called “Student hut” which aims to put students in touch with one another and to ensure as best he can that those who are new to university life are not completely abandoned. I have added a few comments to the sketchbook above which indicates what I think Renton should have said in response to the criticism of University teaching/tutoring. And a bad tutor may not necessarily be a bad researcher and vice-versa. there is room for the most appalling people on University staffs! It is just a matter of finding out where best to pl;ace them, but listening to student feedback is helpful and the Government does not have all the answers or indeed bear all the responsibility for the current mess.
In 1972 I wrote to Richard Williams and was invited to visit his studio. I think I also sent him some artwork. It would have been about this time that a kind lady was also trying to arrange an exhibition for me I learnt recently, so I imagine I was doing fairly well as a little 11 year old draughtsman and impressing more people than I realised. Certainly, it took years to get back to the dynamism and accuracy of those early days partly because I was consistently bullied by the art teacher at school. I was called names by this man, had my work ripped up by him, and was consistently slapped down with words like “slick” and “easy” which I understood then to be criticisms but which today I would accept as some sort of defiant badge of honour. Anyway, this is not meant to be a whine, but more an excuse to think about what I got out of this process. The most important thing is some sort of resilience and determination to keep going whatever happens – that is useful for any animator as Williams testifies. Also, I think this is the source of my interest in teaching- the subject matters not a bit as long as I think I can master it before the lessons begin incidentally, but the desire to ensure that no child is treated as I was is fundamental. Many children are talented. We,as adults, need to harvest or harness those talents.Talent is not really like a plant. It may survive but it certainly does not thrive or grow if you throw alot of shit at it!
I had seen Williams on a childrens’ tv programme called “Clapperboard” and he was shown drawing one of the brigands laughing. I loved the way that the character moved as he laughed. It was subtle and in close-up, but there was clear movement and character. The laugh was something he had recorded himself and I believe, now, that the animation was loosely based on one of the imps in Sleeping beauty. But there is no disguising the mastery. The hand movement on this sequence as the brigand laughs is exquisite and I have looked at it in some detail- Williams draws hands like no one else. This character is now one of many brigands in the second half of the “Thief”.
The sequence is marked by a change from pen and ink outlines to wax pencil outlines that were used also on “Christmas Carol”. At the studio, I was given one of these pencils and some cel and told to draw something which I did, but I was very nervous and I found it difficult. The waxy pencil is easily smudged and is only truly bonded with the cel when it is exposed to hot light under the camera. I should imagine though that the cameraman was forever cleaning the glass panel that holds the animation in place. I think I may have tried using some paint. I am not sure, but I got to use paint later on working for “Wicked Witch” in the late 1980s as they wound up work on “Roger Rabbit” and took on project after project that aped the animation/live action combo style, or simply tried to look computer-generated ( some of the Waterboard adverts that accompanied one of the waves of Thatcher privatisation, for instance which were all actually drawn in coloured pencils on cels that had been sprayed with a formula that made them sufficiently textured to accept the crayons. The same method was used in the Snowman, Father Christmas and the Beatrix Potter films at TVC)
My trip to 13 Soho Square was a day that must have changed my life or at least given it proper direction: in the evening, so excited was I that I vomited with gusto on the train and over my mother’s handbag. I knew then, maybe from some kind of Rorschach test, that I had a vocation to draw animated films. I remember meeting the great man on the stairway in front of what must have been one of his own oil paintings. I draw no parallel at all between my vomit and his painting though I have no real memory of the visual content of either. His picture all looked very dark and grand to me. Animators upstairs flipped scenes that I think I knew even then were from the projected film of “Nasruddin”- I am pretty sure that I saw the thief bouncing from one canopy to another. that was also in the finished print we saw on Sunday. I had seen pirate versions of this on youtube and the australian DVD where it seemed a bit repetitive. In the NFI theatre, with a crowded audience, it looked wonderful. This is broad slapstick and it always needs an audience to get the most out of it!
Later, I went back to the studio a few times and had a delightful dinner with Richard Williams in which he compared computer people to madmen trying to sell crutches to people who have no difficulty walking. “But my crutch is gold plated” he said they would say. “Why walk when you can hobble with a crutch?” This was the infancy of Computer animation and within less than 10 years I myself would be involved for a brief period in the production of computer games animation. But he is right: there can be no short-cuts and nothing replaces the raw knowledge of being able to draw exactly what you can imagine in your head.
I was particularly keen for Necati to see “the Thief” in the best possible way. I have some publicity material the studio gave me by which time the name had changed from “The thief who never gave up” to “Once”.
During the talk after the screening, when a few odd people, one of whom I am afraid I have drawn above, hogged the microphone and went on and on (and on!) about pirate versions of the thief that they had seen on the internet (no one mentioned Gilchrist by name- why not? though Dick Williams urged him to get on with his own work instead of obsessing about “the thief”), Williams talked a bit about his current project,apparently based on “Lysistrata” and called “If I live”. When we met for Dinner in ’82 or ’83, he had been talking about an adaptation of the Epic of “Gilgamesh”- a story about a babylonian Noah figure, and there is a creation account in “Gilgamesh” which lies behind the first creation story in the bible. It is more vivid and much more fun, certainly worthy of animation as indeed is Aristophanes’ “Lysistrata”. I will dig out my own animated versions of Aubrey Beardsley and maybe the (unpublished and scurrilous) comics based on Greek texts and post them on this new blog in time but I suspect Williams is doing his own thing with the Greek comedy and has moved some distance away from Beardsley. I moved from Beardsley too: it simply took up so much time! I would love to know what happened to Gilgamesh and what Williams’ “Gilgamesh” would have looked like and also I would like to know what role the laughing camel must have had in “Nasruddin”. There was alot of publicity about the camel but he makes a very brief appearance in “The thief”. Had the hogs stopped talking evasively about Gilchrist, then maybe I could have asked about Gilgamesh or the Camel. Now, we may never know!! I will write more on this subject another time. In the meantime, here are some sketches made on Sunday afternoon during and after the screening.
I have finally got round to some trial animation for my proposed film of “Trial by Jury”.
animate-tim.com 