Silverlight Mass Communication Project
Well, one of the bright ideas I had for our wedding, was, considering Veronica has a lot of relatives on the other side of Canada, to have sort of a conference call like application to allow her long distance relatives to view the wedding and perhaps even be able to leave recorded responses or ideally interact with guests actually at the wedding. Works with the fact that I'm a programming nerd :)
Anyway, after a bit of research it turns out that having Silverlight interact with the camera is easy. Having it encode and stream data via a web service on the other hand... not so easy. I really have to say, I'm a bit surprised and somewhat disappointed at the lack of support. There are a few hack-ish solutions out there, and obviously some alternatives, and then there are the "Alex" solutions.
So I thought I would write a blog post to consolidate some of those thoughts. Basically, the difficult part in not having an encoder and an easy means of doing all of this is that firstly, there isn't an easy way of doing this and secondly, the lack of a good encoder means handling multiple streams will devastate even a fairly powerful server. The one hack approach I found reported a 5 fps solution chewing up 30% of his server time. This I am assuming was a 2 way chat application. Not a very good framerate to begin with, and since I'm considering much more than 2 camera feeds this already bad performance will degrade fairly quickly.
So, how do I take this and make it better? Well the first thought was to use some fuzzy logic, some multi-threading and some client side encoding to attempt to offload some of that work from the server therefore improving upon the existing hack. Basically, I would build up a queue on the client side of stream data broken into chunks that needs to be sent to the server. If the server is behind and the assumption is that it will be at some point, then in a separate thread start encoding frames further back in the queue. So, when my queue reaches a size of N chunks, I start encoding everything from the Nth chunk onward on the client side while I wait for the server responses to catch up. When it gets time to transmit these pre-encoded chunks, a flag will indicate to the server that it doesn't need to do the encoding, the chunks will already be smaller since they were encoded and so upload times should be shorter AND they will alleviate the encoding time on the server allowing it to catch back up. The fuzzy logic comes in at determining what the value of N should have based on current server response times.
Basically, since the client will encode much more slowly than the server, we only want to start encoding if we think we can begin and end encoding the chunk before the server is ready to start receiving it.
Option number #2 is to steer away from Silverlight. Just do a WPF application and then each person connected can do their own encoding as a stand alone application will have far more CPU cycles available for encoding than a Silverlight client application. Then it can communicate directly with a core server application that merely redirects the streams to the appropriate clients. Probably the easiest way from a coding perspective to get fair quality in a live feed, but makes it more difficult for the end users as they now have an application they need to manually install.
Option number #3 is to not aim for live feeds at all. Shoot everything in "Chapters" and as each chapter is successfully encoded it is published for others to watch. User comments could be uploaded in a similar fashion. Downside is that there would be neither live feeds or face to face communication (at least not via my application, could perhaps have a Skype account in use for that).
But is live really needed? Probably not. Veronica's point was that a live feed might be inconvenient anyway. We won't want to be chained to a computer as the bride and groom. We will probably want to be moving on to pictures shortly after the ceremony, etc..., always moving. We want people from long distance to be able to view the wedding, and we would also love to be able to have them send their well wishes and whatnot, but neither of those needs to be live. And with the emphasis on recording and encoding in perhaps higher quality, feeds retrieved in this fashion would be better quality for wedding video.
Will discuss with my bride to be, and come up with a decision, but for now, I have the options out there and can better ponder over them.
Anyway, after a bit of research it turns out that having Silverlight interact with the camera is easy. Having it encode and stream data via a web service on the other hand... not so easy. I really have to say, I'm a bit surprised and somewhat disappointed at the lack of support. There are a few hack-ish solutions out there, and obviously some alternatives, and then there are the "Alex" solutions.
So I thought I would write a blog post to consolidate some of those thoughts. Basically, the difficult part in not having an encoder and an easy means of doing all of this is that firstly, there isn't an easy way of doing this and secondly, the lack of a good encoder means handling multiple streams will devastate even a fairly powerful server. The one hack approach I found reported a 5 fps solution chewing up 30% of his server time. This I am assuming was a 2 way chat application. Not a very good framerate to begin with, and since I'm considering much more than 2 camera feeds this already bad performance will degrade fairly quickly.
So, how do I take this and make it better? Well the first thought was to use some fuzzy logic, some multi-threading and some client side encoding to attempt to offload some of that work from the server therefore improving upon the existing hack. Basically, I would build up a queue on the client side of stream data broken into chunks that needs to be sent to the server. If the server is behind and the assumption is that it will be at some point, then in a separate thread start encoding frames further back in the queue. So, when my queue reaches a size of N chunks, I start encoding everything from the Nth chunk onward on the client side while I wait for the server responses to catch up. When it gets time to transmit these pre-encoded chunks, a flag will indicate to the server that it doesn't need to do the encoding, the chunks will already be smaller since they were encoded and so upload times should be shorter AND they will alleviate the encoding time on the server allowing it to catch back up. The fuzzy logic comes in at determining what the value of N should have based on current server response times.
Basically, since the client will encode much more slowly than the server, we only want to start encoding if we think we can begin and end encoding the chunk before the server is ready to start receiving it.
Option number #2 is to steer away from Silverlight. Just do a WPF application and then each person connected can do their own encoding as a stand alone application will have far more CPU cycles available for encoding than a Silverlight client application. Then it can communicate directly with a core server application that merely redirects the streams to the appropriate clients. Probably the easiest way from a coding perspective to get fair quality in a live feed, but makes it more difficult for the end users as they now have an application they need to manually install.
Option number #3 is to not aim for live feeds at all. Shoot everything in "Chapters" and as each chapter is successfully encoded it is published for others to watch. User comments could be uploaded in a similar fashion. Downside is that there would be neither live feeds or face to face communication (at least not via my application, could perhaps have a Skype account in use for that).
But is live really needed? Probably not. Veronica's point was that a live feed might be inconvenient anyway. We won't want to be chained to a computer as the bride and groom. We will probably want to be moving on to pictures shortly after the ceremony, etc..., always moving. We want people from long distance to be able to view the wedding, and we would also love to be able to have them send their well wishes and whatnot, but neither of those needs to be live. And with the emphasis on recording and encoding in perhaps higher quality, feeds retrieved in this fashion would be better quality for wedding video.
Will discuss with my bride to be, and come up with a decision, but for now, I have the options out there and can better ponder over them.
Comments
Post a Comment