Dev Journal: December 14th 2018 - Server side looking good.
So, my project is coming together incredibly. At least, from the backend perspective. It is FAR from finalized and lacking a lot of polish. But, the successes are hard to question.
Basically, I deployed RabbitMQ and started projects using the dotnet core clients. Wrote some boilerplate code and base classes to make leveraging topic exchanges for communication and direct queues for logging simple. Then, I built some abstract classes which define some basic handler types and a lot of the most basic functionality. Then I implemented a few different implementations.
Early on, I had a problem with zombie threads, which were in turn scuttling my access temporarily to my Nest API calls. I fixed those easily enough. Then, I needed to extend the PoC far enough to prove out the server side of things.
The two abstract classes most of it is built off of are what I'm calling a Scheduler and Triggers. Schedulers do MOSTLY what you think. But the general concept is actually much lighter weight. The fundamental job of these schedulers is not to do any work directly (though they can).
But, rather they simply publish to one of my named exchanges (typically Request) with a configured Topic, and payload (possibly empty if the existence of a request is enough). The topics are named.., so, to trigger reading my thermostat, I use a topic like this Temperature.Read.Nest. This way, I can easily build consumers for these topics. And, by default it posts back to a topic with the same name, but on my "Response" Exchange.
This is far from the final for even this simple class. I want to implement a base "Payload" class which allows the requestor to be more specific about how to respond, to provide meta data about the payload and maybe some metadata about the requestor. This would enable a requestor to tell whoever responds to the request, for example, to respond, not via an Exchange with multiple subscribers, but perhaps to a direct queue instead. And, allowing some metadata about the payload could allow for more robust receivers.
The trigger class represents a class which is typically listening on a particular topic in the Request Exchange. When a queue it is bound to gets a message, it processes the payload, performs some work, and generally responds back on the Response Exchange.
So, I create a basic scheduler from my abstract scheduler which, every 30 seconds posts to the Request Exchange for Temperature.Read.Nest.
I then made a trigger which listens on Temperature.Read.Nest. Every time it receives a request, it makes an API call to Nest, from which I can get data like the current temperature, target temperature and so on, it then converts it into my internal TemperatureReading class, serializes it and posts it back to the Response Exchange.
Next, I made a logging trigger to store the data. When it sees that response, it writes a new entry into an SQLite table which I can use for graphing historical data or for driving other decision.
Lastly, I wrote a scheduled task which overrode my base scheduler, so instead of publishing out to an Exchange, it simple cleans out all records in the TemperatureReading table older than a given amount of time.
Because most of the code is in the abstract classes, my actual implementation contain as few as 15-50 lines, with many of those being lines containing whitespace or open or closing braces. I don't need to keep solving problems like zombie threads or recreating the logic which actually creates the publishers and subscribers. I'm successfully running multiple threads on multiple exchanges topics and queues. And, since I have a test program running in the background, I even have multiple consumers for my exchanges and queues.
I still need a lot. Right now I'm spinning up hard coded configurations. There needs to be some configuration somewhere and better yet, a UI. Which means more code to manage that. There is also what I mentioned about abstracting the classes used in the Exchanges to provide more flexibility, control and data. And then, then I need to figure out how I want to handle UI and then get Rabbit working there as well. UI will probably be JavaScript/HTML based. But, there are RabbitMQ clients for that already. And, it could always be something like ASP.Net instead, since I AM already using that for the service layer.
I think I also want my next set of plugins to talk to openHab. It is a pretty quick way to get support for a broad range of smart home devices.
I may also split out my dedicated plugins like the Nest one even further. The 1 API call I get right now returns a decent amount of data about ALL of my devices. But, I'm only using it at the moment for the thermostat. When I start integrating Nest Cam support, I'm going to want to cache that data off and re-serve it up to requestors and only update it once, on a predefined schedule. At present, that integration is very much hard coded for my needs.
Of course, I think that is the first stage. Build what works for me first. Generalize it later.
Basically, I deployed RabbitMQ and started projects using the dotnet core clients. Wrote some boilerplate code and base classes to make leveraging topic exchanges for communication and direct queues for logging simple. Then, I built some abstract classes which define some basic handler types and a lot of the most basic functionality. Then I implemented a few different implementations.
Early on, I had a problem with zombie threads, which were in turn scuttling my access temporarily to my Nest API calls. I fixed those easily enough. Then, I needed to extend the PoC far enough to prove out the server side of things.
The two abstract classes most of it is built off of are what I'm calling a Scheduler and Triggers. Schedulers do MOSTLY what you think. But the general concept is actually much lighter weight. The fundamental job of these schedulers is not to do any work directly (though they can).
But, rather they simply publish to one of my named exchanges (typically Request) with a configured Topic, and payload (possibly empty if the existence of a request is enough). The topics are named
This is far from the final for even this simple class. I want to implement a base "Payload" class which allows the requestor to be more specific about how to respond, to provide meta data about the payload and maybe some metadata about the requestor. This would enable a requestor to tell whoever responds to the request, for example, to respond, not via an Exchange with multiple subscribers, but perhaps to a direct queue instead. And, allowing some metadata about the payload could allow for more robust receivers.
The trigger class represents a class which is typically listening on a particular topic in the Request Exchange. When a queue it is bound to gets a message, it processes the payload, performs some work, and generally responds back on the Response Exchange.
So, I create a basic scheduler from my abstract scheduler which, every 30 seconds posts to the Request Exchange for Temperature.Read.Nest.
I then made a trigger which listens on Temperature.Read.Nest. Every time it receives a request, it makes an API call to Nest, from which I can get data like the current temperature, target temperature and so on, it then converts it into my internal TemperatureReading class, serializes it and posts it back to the Response Exchange.
Next, I made a logging trigger to store the data. When it sees that response, it writes a new entry into an SQLite table which I can use for graphing historical data or for driving other decision.
Lastly, I wrote a scheduled task which overrode my base scheduler, so instead of publishing out to an Exchange, it simple cleans out all records in the TemperatureReading table older than a given amount of time.
Because most of the code is in the abstract classes, my actual implementation contain as few as 15-50 lines, with many of those being lines containing whitespace or open or closing braces. I don't need to keep solving problems like zombie threads or recreating the logic which actually creates the publishers and subscribers. I'm successfully running multiple threads on multiple exchanges topics and queues. And, since I have a test program running in the background, I even have multiple consumers for my exchanges and queues.
I still need a lot. Right now I'm spinning up hard coded configurations. There needs to be some configuration somewhere and better yet, a UI. Which means more code to manage that. There is also what I mentioned about abstracting the classes used in the Exchanges to provide more flexibility, control and data. And then, then I need to figure out how I want to handle UI and then get Rabbit working there as well. UI will probably be JavaScript/HTML based. But, there are RabbitMQ clients for that already. And, it could always be something like ASP.Net instead, since I AM already using that for the service layer.
I think I also want my next set of plugins to talk to openHab. It is a pretty quick way to get support for a broad range of smart home devices.
I may also split out my dedicated plugins like the Nest one even further. The 1 API call I get right now returns a decent amount of data about ALL of my devices. But, I'm only using it at the moment for the thermostat. When I start integrating Nest Cam support, I'm going to want to cache that data off and re-serve it up to requestors and only update it once, on a predefined schedule. At present, that integration is very much hard coded for my needs.
Of course, I think that is the first stage. Build what works for me first. Generalize it later.
Comments
Post a Comment