Roundtable: The Quantum theory of storage
TVBEurope hosted a roundtable in partnership with Quantum on the future of storage for broadcasters. What became apparent was that storage problems are rarely technological, they’re almost always cultural
Quantum's TVBEurope roundtable last week decided that a good storage solution first requires a good workflow solution
In November, TVBEurope hosted its latest industry roundtable at London's Soho Hotel. Held in association with Quantum, the afternoon's theme was storage for the broadcast industry - what are broadcasters' current storage needs, what storage solutions will new media technologies require, issues of data protection and the impact of cloud technologies.
Participants in the conversation were BT Sport innovation producer Daragh Bass, Sony head of IT and workflow solutions Niall Duffy, Viacom senior director of broadcast and production technology Rod Fairweather, BBC Sport head of post production Lawrence Windley, with moderation by Jeremy Bancroft of MAC, and Quantum's regional sales director, Christo Conidaris.
What quickly became clear in the free-ranging discussion was that you can’t make decisions about your storage needs without a clear understanding of your workflow.
Pushing it to the limit
Every one of the roundtable participants cited the problem - so ubiquitous it's a cliche - of ballooning file sizes and absurd shooting ratios pushing storage capacity to its limits. It's a truism of modern life that no matter how much storage you have, you will always fill it.
Quantum's Christo Conidaris noted that the media industry has a unique propensity for gobbling up storage. Quantum supplies storage products for a range of sectors, from biotech to government to fossil fuel companies. Conidaris noted: "One thing that's interesting about the media and entertainment industry is you guys really push technology to its limits, whatever you're doing, whether it's storage or cameras or networks. We've found that the adoption rates for new technologies is usually higher in the media and entertainment than in other industries."
His statement surprised the group, who seemed to feel that their organisations were very conservative in adopting new technologies - slightly proving Conidaris' point. No matter how much tech you have, in the TV industry, it's never quite enough.
But unlike other industries, the final product at the end of millions of TV euros spent is a set of files on a drive. And with data itself the final fruit of everyone's hard labour, it's no surprise that broadcasters want to hang on to as much of it as possible for as long as possible.
Costs of storage are coming down (although Sony's Niall Duffy suggested the "Brexit effect" is reversing that in the UK, with storage prices rising since the June vote), but Viacom's Rod Fairweather pointed out that the endless hunger for storage negates any economic benefits: "The cost goes down on storage, but the amount we're producing goes up, so we never get the benefit of this stuff getting cheaper."
Technology – in every sphere of our lives – allows for greater speed of deployment and amplified leveraging of an organisations skills, and has allowed fast deployment of new companies and new projects, but the agility conferred by technology has sometimes led to an abandoning of methodology. The dream that technology will solve what are essentially issues of planning and thinking inevitably boomerangs to create a host of difficulties later.
“When broadcasters talked about being ‘agile’ as a method, they rushed into things,” noted Duffy. “But what we’ve realised is that the sheer joy of being able to start small and focused has now resulted in: ‘Oops, actually we didn’t structure it properly’. And we’ve let it expand to a point where it’s no longer what we really wanted, because we didn’t think about what we wanted. It’s like building a house without a foundation, because we wanted to start quickly on the ground floor.
“We’re a very immature industry in many respects. Our problem with archive and library is that we don’t know what we’re going to do with it, because we’ve done none of the analytics, we don’t know how much it costs, and we’re terrified of losing something.”
"It's always been an issue ever since the first videotape was created: nobody's good at housekeeping," said Viacom’s Rod Fairweather. "Nobody ever gets thanked for chucking something out. But you can get fired for chucking something out that shouldn't have been chucked out. Whoever threw out the original Doctor Who's was vilified for life. Until we change that culture, no one's going to risk their jobs by unnecessarily deleting things."
One reason for the vagueness around storage is that it’s not often clear who is paying the bill. The roundtable observed that if long-term storage was charged to a production rather than being absorbed by the broadcaster, you would see ballooning shooting ratios drop dramatically, and thrifty shooting would be the order of the day. If a production isn’t paying for storage, their mindset is that the storage is free.
“A producer may insist that he wants to bring back all 3000 hours of content he’s shot,” noted Duffy. “But when you analyse it, the long-term cost of that is tens of thousands of pounds per year.”
“But If you charge that back to an actual production, they’ll be very quick to say: ‘Oh, no. We don’t need that any more,” quipped BT Sport's Daragh Bass.
“And if you do decide to keep all that, there’s the cost of the time it takes to look at the material and decide what has value,” added Viacom’s Fairweather. “That adds hugely to the costs of a productions.”
BBC’s Lawrence Windley believes that production teams haven’t really absorbed the reality of everything digital storage implies.
“As an industry the production side of the industry hasn’t caught up with the long tail that these things have. You’ve done your online edit and conform and had your cappuccino in Soho, and you think it’s done then. And in the days when you would then go home with a box of tapes, that was it. But now, that digital content sitting on servers somewhere still needs to be catalogued and looked through. With us, there are time pressures. The production teams go on to the next project and no one has the time to sit with an archivist for four or five days.“
Moderator Jeremy Bancroft added that in many cases, those teams not only aren’t available to sit with an archivist, but they’ve entirely disbanded, having been brought together solely for a single production: “And you can’t find that knowledge and skill (of an archivist) anymore. If it isn’t captured at the time of production, it’s probably lost forever.
It’s the workflow, stupid
Windley confirmed that beyond the deceptively simple issue of "where to keep your stuff" lies the foundation of a broadcaster's workflow - its entire production ethos. "It's amazing how much we've talked about workflow, even though we started with talking about storage and library and archives. It very quickly goes into a conversation about workflow and decisions."
"We still haven't really thought through the kind of workflows that we need," said Sony's Niall Duffy. "We're still in a world where we inevitably come back to technology choices. That's the way the industry thinks, with the creatives often leaving it up to the engineer to come up with a solution. And the engineer gets quite excited by the technology rather than the process. But I think we're finally beginning to see that change, with companies like SDVI who are talking about supply chain management. When you've got a clear view of what your supply chain is and how you want to manage it - which every other industry outside ours does - it doesn't become an issue of technology choices. Instead it's: What do I need?"
BT Sport's Bass is looking for alternatives to on-premises storage. He sees an opportunity in rights holders taking responsibility for the storage of all assets related to project with broadcasters taking over only at the playout stage: "There are more opportunities where the rights holder is also the distributor and you don't store anything locally. You pull it down as you need it and they manage everything on your behalf. You have access to it now, but in a week's time, say, your access runs out."
Bass said BT Sport had the advantage of building being able to build storage workflows from scratch when it started in 2013. "We decided we needed rules from day one. And they were rigid. We had really strict deletion rules for all the different types of content. It was a 'keep' decision, rather than a 'delete' decision."
Fairweather said that the needs of Viacom, which oversees a wide variety of stations, and the producers that support them must navigate a complex path in content storage.
“It depends on what type of editing you’re doing, what scale, what timeframe. Our workgroup sizes are all very different. A cloud environment may be more appropriate for storage if you’re working in a collaborative environment, but there is no one correct storage solution. And that’s one of the reasons making decisions about storage is so difficult. There is not one answer. We have promo producers sometimes working individually. Some of our workgroups in small regional stations may not have a lot of technical support. Or we may be working with a big Soho company. Our solutions are much more complex than how big, how fast.”
Resistance to MAM
Even given the obvious need to organise workflows and storage, content owners are often willing to make short cuts on tech that doesn’t have an obvious relationship with what ends up on screen. With everyone with an internet connection becoming a potential broadcaster, there are a million headaches in the making for all those who attempt mass storage and archive without a proper asset management system or workflow in place.
At the end of the day, for the TV industry, cost of operations is tiny compared to the cost of rights. So we still have this model of even if it seems expensive, it’s not when compared to rights. But compared to other industries, the things we do are extraordinarily expensive. So if you look at a publisher – and there are so many of them trying to get into video - they don’t really know who to speak to, so they’ll go to a broadcast systems integrator and they will literally fall off their chair: ‘How much?! Asset management’s not free?’”
“Sometimes it’s very difficult to find the money for those things,” added Bass. “They may be buying a lot of cameras and high-end gear and ambition, but when it comes to buying an asset management system the question is: ‘What will I see onscreen for that?’ Well, nothing. But it will manage all your content. ‘Can we not do that in Google Drive?’.”
But Niall Duffy said that some big broadcasters were no less stingy: “To be fair, I’ve seen so many broadcasters cave in on that too, rather spending the money to do a MAM properly, because they can’t see an ROI on MAM. And it’s not that they can see an ROI on storage either, but they’ve got production screaming at them saying ‘We have to have this!’. They’re not saying ‘We have to have a MAM.”
Fifty years ago, most of media storage consisted of cans of film and reels of tape in boxes on shelves with paper tape labels. If you wanted something, someone had to physically retrieve it with their hands.
Now, media can be stored almost literally anywhere. Often, as in the case of cloud storage, it’s stored in no one place in particular – a bit like the electron only deciding on a location still once you want to access it. How appropriate that it was Quantum hosting a roundtable on a subject which has the infinite scalability and infinite variability that mirror something as hard to pin down as particle physics.
We are entering a phase in which the “where” of storage is becoming increasingly meaningless. Less and less does media is media stored in a real place in any practically meaningful way and as a result the conversations around storage become increasingly esoteric, until media storage begins to look like an arcane philosophical discipline.
What the Quantum roundtable made clear is that the way forward for media companies is not: Where do I keep my stuff? But: For what purpose am I keeping my stuff? And how do I best evolve my workflows and my mindset to secure that purpose into the future.
“One thing that's interesting about the media and entertainment industry is you guys really push technology to its limits, whatever you're doing”
“"The cost goes down on storage, but the amount we're producing goes up, so we never get the benefit of this stuff getting cheaper”
“We still haven't really thought through the kind of workflows that we need”