EPISODE 9 | Guest: Ellyce Kelly, client relationship manager for Visix
Running a digital signage pilot is a vital step for every installation, no matter the size. Whether you’re launching a new system for the first time, or adjusting one that has changed or grown, you need to be able to see how things will go before you “go live” in front of your audience.
It’s better to make a few mistakes during a pilot than when the audience you’re trying to impress is watching. Consider it something like a dress rehearsal. Whether you’re trying out new software for the first time or overhauling your look, a pilot gives you a chance to launch, measure and adjust without alienating viewers.
However, there are right and wrong ways to approach a pilot. You need to make sure to include all stakeholders, have measurable goals and reasonable timelines. Doing it half-baked can be as bad as not doing it at all.
- Understand the benefits of running a digital signage pilot
- Get an overview of the pilot process
- Explore different types of pilot goals
- Learn how to assemble and train your team
- Review how to assess and adjust
Subscribe to this podcast: Podbean | Spotify | Apple Podcasts | YouTube | RSS
Learn more about this topic in our Masterclass Guide 1: Digital Signage Systems Overview
Transcript
Derek DeWitt: Now you wouldn’t ride a roller coaster that hadn’t been tested, would ya? All right, maybe that’s a little extreme, but maybe a better example is rolling out a website – you’d look at every page, you’d make sure that everything works, you’d get some feedback, and you’d probably do a lot of fine tuning and correction before you went live, right?
So, exactly the same thing when you’re doing a digital signage deployment. Today, we’re going to talk about how important it is to run a pilot before you go live or to run a pilot when you’re going to make any major change. I’m here with Ellyce Kelly, client relationship manager for Visix. Hi Ellyce, how are you?
Ellyce Kelly: Hi Derek, it’s great to see you again.
Derek DeWitt: So, I have a list of steps and phases in running a pilot. You’ve got this software and you’ve got your system and you think, “Oh, I’m just gonna throw it up there.” Big mistake, right? You need to run a pilot first. So first off, what is a pilot?
Ellyce Kelly: So, a pilot is typically a 30- to 60-day dry run of your initial digital signage deployment. So, you’ve got to get all of your stakeholders in place. You need to figure out, you know, what do we want to achieve with this? We don’t want to just roll everything out and say, “Oh, we’re going to put all this content on these screens and these screens and these screens” – that never works. Right? So this is a good way to start.
Derek DeWitt: Do people who already have a system up and running, do they use it sometimes, like if they’re introducing a new element or they’ve changed their workflows or they want to evaluate some other aspects – they can also run another, like a sub-pilot later on? You know, like, “I’ve had the system for a year; you know what? I want to know if I can…how does this work, how does this workflow work, where can we improve this?” Something like that too, right?
Ellyce Kelly: Sure. Absolutely.
Derek DeWitt: What are the benefits? Why do this? It seems like an awful lot of work.
Ellyce Kelly: What’s going to be more work is if you don’t do this. So, it’s going to lower the risk of you… you’re evaluating your goals, you’re evaluating your resources in this controlled space, so you’re not on this production live. I mean, that’s not where you want to try that, right? So, you can also increase your stakeholder buy-in, so it gets more people involved. That’s another thing that is important.
And then you get your designs, right? So, let’s say you’ve got this new look, or you want to roll out a new look, you don’t want your screens to all look the same and have the same content. What do you want them to look like? That’s one of the hardest decisions, it is, for folks to make. “What are we…? I don’t know what we want. Well, let’s find out what we want, and then that way we can roll it out in a smaller scale.”
Derek DeWitt: Version one, version two, version three…actually, I liked two the best….
Ellyce Kelly: Yes. And then tweak it as needed. You can also test data sources and external feeds, which is great, another reason you would do this.
Derek DeWitt: So, if you’re like pulling in… like you just bought a bunch of content subscriptions or something like that. “Hey, how do they work?”
Ellyce Kelly: XML, event data, Excel… if you’re pointing to other sources, you want to get other data on the screens that you haven’t done before, that’s a great place to do that. And then you can also see, you know, who is a leader and who you can use for maybe more training or advice.
Derek DeWitt: What are some overall general tips for pilots?
Ellyce Kelly: Well, you definitely need to have some goals and a plan before you begin. So, who are your stakeholders? Who’s going to be involved in this pilot? What are you trying to accomplish? You know, what features, what capabilities, what are you hoping to get out of the evaluation itself?
Also, do you want to…what content do you want on your displays? Do you want schedules, like events I was saying, do you want to point to an XML, Excel, do you want weather? I mean what are we showing? What do we want things to look like? Do we need to have audio? If you have it in an ER waiting area, you may want to have it or maybe in an urgent care facility, I mean it really depends. Also, if you’ve maybe got a queuing area that you might be using it. It just depends on the location. But that is again, that’s something you just have to consider.
And then, who should be involved? I mentioned that earlier. And then who’s going to be configuring, implementing, maintaining the system. And then who’s going to be creating, approving and managing that content? That’s a big one.
Derek DeWitt: Because often that’s not the people who bought it.
Ellyce Kelly: Not the people who bought it, exactly. Because you also might just be trusting of something, you know that works on your network or do you need a play or relay, or do you need to add this other element in?
Derek DeWitt: So, the first thing you do is “set goals”. What do you mean?
Ellyce Kelly: So, just really, testing all the components on the network. So I just mentioned that, that last piece there…
Derek DeWitt: So, that could be a goal.
Ellyce Kelly: That could be a goal.
Derek DeWitt: So, the goal is like – why are we doing this pilot? It might be, like you say, it was maybe the first time we haven’t actually deployed live yet. Does it work? Let’s test it.
Ellyce Kelly: And some pilots are that short and to the point. They want to see if something will run on their network with all of their security settings and everything else. And that pilot could be over in 30 days, might be over in a week.
Derek DeWitt: What else? What other sorts of goals might there be?
Ellyce Kelly: So, configuring third-party applications on your digital signage system. If you’re talking to external parties, third-party systems, we need to make sure that we can reach all of those again, perhaps in your environment, network environment.
Also choosing layouts and layout schedules. A lot of folks will just maybe think of an initial layout – that’s going to get boring after a while; not even after a while, pretty quickly!
Derek DeWitt: Day two.
Ellyce Kelly: Day two, exactly. So we love layout schedules, and that’s something that gets you also thinking about that content of what all do we need and who all needs to be producing, how much of that is going to be automated, how much is going to be, you know, created in-house.
And then also just defining the types of content that work best on specific displays. Just like we were just talking about with audio. Some screens may need a little audio, some may definitely not need audio, where you have maybe high traffic or you have someone sitting in an area where they’re helping students, they don’t want to hear those…
Derek DeWitt: Library.
Ellyce Kelly: Exactly. Nobody wants to hear that there.
Derek DeWitt: And again, very often, sometimes like a library you can probably figure out, but other locations you might not know until you run a pilot to see, “Gosh, you know what? It turns out, I thought that was a good idea; it’s really irritating.” Workflows and processes – could that be a purpose?
Ellyce Kelly: Absolutely, those absolutely play into this. Also, I think maybe evaluating the teams – who are the teams, who’s going to be contributing to this? So, a lot of times folks will get, “Hey, guess what? We’ve got digital signage. I know that you’re really busy and you have a full schedule, but you’re also now – this is your new task. You’re going to be creating content and putting content on the screens.” So, let’s go ahead and get those folks involved, determine who that’s going to be, so that they can also contribute to this pilot.
Derek DeWitt: And they can then actually go through a real-world, sort of a mock real-world, period of time to go, “Oh, that’s how it works!”
Ellyce Kelly: And then you’re also going to want some feedback on that. So, you need to determine effective methods to measure your audience engagement. It’s not just about getting content on the screens – are people looking at the screens?
Derek DeWitt: Are clients actually bothering to measure audience engagement, as a general rule, in your experience?
Ellyce Kelly: Not when I first talk to them, but they are definitely thinking about it once I’ve talked to them.
Derek DeWitt: It’s important…
Ellyce Kelly: It is. A lot of customers are very excited about that. They think, wow, that’s a really good idea. And I think when they understand that they’ll be able to take that information to their management team and say, “Hey, we’re getting some great return on engagement on this system that you bought, you know, to meet these goals. We’re meeting them, and we can show you proof of that.”
Derek DeWitt: Really at any point…I mean you could have a digital signage system for five years and once you make any major change, you’re gonna run a pilot.
Ellyce Kelly: Sure.
Derek DeWitt: So, tell me about…”define the scope” is what I have on my list here. What does that mean? Is that like parameters or what is that?
Ellyce Kelly: So, really it is parameters – where the pilot will run. Are you going to run it in your marketing department or you’re going to run it on one screen? Are you going to, you know, where’s it going? Is it going in break rooms or is it going to go in the lobby?
I mean you really have to determine…and a lot of times it might be at a marketing or communications desktop, like they might set up a player in a display right next to the marketing stakeholders.
Derek DeWitt: It can be that simple?
Ellyce Kelly: It can be that simple. Or it could be in a more production environment where you’re trying to get a lot of feedback and a lot of measurement from a lot of different people.
Derek DeWitt: And how you do that really depends on what goal you’ve set.
Ellyce Kelly: It really depends on your goals, it does. And then you also want to come up with a timeline for the pilot. You don’t want it to just run endlessly, but you want to have a timeline for the pilot, the evaluation and also some adjustments which we’ll talk about a little bit later. But you definitely have to make some tweaks possibly to your pilot and you need to account for that in your timeline.
A list of action items, with task owners and deadlines – timeline, very important. You definitely want to make sure that you have that.
Derek DeWitt: Once the pilot’s done, you don’t have infinite time to tell me the results.
Ellyce Kelly: You really don’t. Everybody wants to keep moving forward. Right? You want to start using it. So, techniques to collect stakeholder feedback. I mean, how are you going to get that feedback? Again, this all depends on how many people are involved.
Derek DeWitt: And I should think also you want to, kind of work out how…so this is our goal that we’ve set, we’re going to determine, whatever, whether or not processes work. How are you going to measure that? Because you know, it’s one of those great counterintuitive things that you actually care about what you measure, strangely enough. So, whatever you measure is actually what you care about, and so you’re going figure out that measuring system, I should think, as well. I would imagine like, how are you going to collaborate together using SharePoint or something?
Ellyce Kelly: Yeah, project management, collaboration, whatever tools you already have in place, whatever you’re using to collaborate. Now you might want to use those. Just like you said, SharePoint might be a great way.
Derek DeWitt: Or one of the other mediums of things like that.
Ellyce Kelly: Exactly. Documentation methods, storage…
Derek DeWitt: Right. Where is it? Where does the documentation sit? How do I access it?
Ellyce Kelly: How do you access it…And then budget and resources.
Derek DeWitt: Oh, sure! Is there any money for this?
Ellyce Kelly: Yeah. If not, maybe not the time to do the pilot.
Derek DeWitt: Oh, you’re doing it on your free time for free.
Ellyce Kelly: Exactly.
Derek DeWitt: All right, so you’ve done all this, you’ve worked out all this stuff, and now it’s time to train your team. And I would guess a lot of people skip this step or forget it.
Ellyce Kelly: Who’s going to need training, you know, during the pilot phase? What features do they need to be trained on?
Derek DeWitt: Right. They may already know some things.
Ellyce Kelly: They may already know some things, but they also, if they’re not responsible for scheduling layouts, we don’t need to teach them that, right? We only need to teach them what they need to know, and then if there’s more advanced things that they need to know later or things beyond maybe what’s in the pilot, we can get all of that taken care of at that time.
Really that’s, you know, do I need technical training, do I need design training, things to consider. And then just what type of content do you want to use in the future? Like, what are they interested in showing? Are they interested in showing content they already have? Do we need to get some content subscriptions?
Derek DeWitt: What are you going to try and expand it later?
Ellyce Kelly: Right. I love to find out – one of the first things that I’ll ask if I’m working with marketing or PR teams or students, “What’s your vision of what’s on that screen at that location?” And it’s interesting when you get a few people from the same team on there, they kind of all have different ideas. So, it’s good to get that worked out…
Derek DeWitt: Which could be problematic!
Ellyce Kelly: It could be if you’re not doing a pilot. So that’s something else that you can iron out in the pilot rather than trying to figure that out on day one of training, live production training or just, “Hey, we all want to do this, what we think we want to do this, but we’re not sure.”
Derek DeWitt: The purpose is to fine-tune and optimize the actual deployment. So, you want things to be as real-world as possible, I should think. The content should be as real-world as possible. I know some companies will just …I can’t remember what it’s called, but you know, the fake Latin text Lipsum – whatever-the-heck it is, you know,…
Ellyce Kelly: Not if you’re doing a pilot with me! And they’ll pilot with me! I actually don’t allow the word “test” in anything. No “test playlist”, no “test images”, no “test, test, test.”
Derek DeWitt: So, it needs to be really, really, like no joke, a simulator.
Ellyce Kelly: I want to see, what content did you make for this event that’s happening on Friday? Because we’re going to go ahead and schedule it, and it’s going to stop playing Friday at four when the event’s over. We’re doing it real-world.
Derek DeWitt: Right. All right, so now, after doing all of this, can we run the darned thing though? Can we run the pilot?
Ellyce Kelly: Absolutely. And all that sounds like a lot of work but it’s really not.
Derek DeWitt: Yeah. Maybe will that take like…it depends on how efficient you are, I guess, but a week, maybe two?
Ellyce Kelly: Everything we just talked about.
Derek DeWitt: So now we’re going to run it.
Ellyce Kelly: Yes!
Derek DeWitt: And see what happens.
Ellyce Kelly: Yeah, for the time allotted. So, whatever the time is that you’ve said. So, you might have 30 days, you might have 60 days, you might have 90 days…
Derek DeWitt: Now why would you…why would there be different time periods?
Ellyce Kelly: Well, it really depends on what your goals are. So, if you’re just testing something to make sure something runs on the network, like we were saying earlier, you might only need 30 days, you might even need five days. You’d get that pre-implementation, the implementation, done that quickly. You can tell real quick if something’s maybe not going to work on your network or not going to work in your environment. And that could be in a very high security environment.
And sometimes that just needs a tweak to how the hardware and the network settings, or how all that’s configured. That’s usually takes a quick, just a tweak. So that could be a short pilot, right? Very quick. But if you’re evaluating design changes and layouts and data sources and mixing in some other more higher-level type eccentric or complex content, or campaigns, then we need to go a little longer.
Derek DeWitt: How good an idea is it to run…like I would imagine some people would say, “Well actually we need to evaluate five things. Can’t we just do them all in one pilot?” Is that a good idea? Is it a bad idea? Does it matter?
Ellyce Kelly: It depends. It depends on what those five things are. But yes, you could evaluate those things as long as you’re focused on them, and you have a way to measure them and you’re not overlapping goals with outcome.
Derek DeWitt: Right. And you make sure that you know who’s responsible for each thing and how they measure the goals, how they report. All right, so now our month or two months or three months, this time period’s over…now what?
Ellyce Kelly: Well, now we’ve got to collect data. We’ve got to collect feedback from our stakeholders. So everyone that was involved, what are they reporting on? Are they reporting on a piece that they were responsible for? Is it a group collaboration? Everybody’s talking and saying, “Well, I just loved it. I love the way this looks. That was so easy. Those layout schedules are a breeze.”
Derek DeWitt: You want like hard data, whenever possible. I want to know – yes, we saw this many more people loitering here….
Ellyce Kelly: Yes, you want that checklist that – literally you’re checking items off and you’ve got numbers; you can fill in the blanks.
Derek DeWitt: So you’d get all this, you evaluate, you go, “Okay, did it meet…did the system perform the way we thought it would? Did the new design layouts, were they more effective?” Whatever your goals were, you do all this and then…
Ellyce Kelly: You tweak.
Derek DeWitt: And then you tweak.
Ellyce Kelly: Yeah. So, I had a pilot not too long ago, um, had some layout designs and had some XML data that they were pointing to…it may have been Excel data…XML, Excel.
Derek DeWitt: One of them!
Ellyce Kelly: Had to change the design a little bit and wanted to change the way the data was being shown. So how often the data was showing. So, there were some tweaks to that that took literally a day to make a couple of changes. When I say “a day”, I mean a quick phone call, and then the next day they’re running their new designs and they’re running the new way the data is loading…
Derek DeWitt: Live…
Ellyce Kelly: Yeah, live. Yes. And so that was, that decision was made the next day, because they said, “Okay, that made a difference!” They were able to measure it that quickly. So, it really just depends on what you’re looking at, what you’re evaluating. But you might, you might have to make tweaks. It’s highly possible.
Derek DeWitt: All right. Wow! So, it sounds like doing a pilot is an awfully good idea. I’d like to thank Ellyce Kelly for talking to us tonight, and thank you for listening, everybody.