First, some context.

I’ve written a program that starts running when I log on and produces data every two seconds. This daemon keeps all the collected data in memory until it gets terminated (usually when I shutdown the system), when it will dump the collected data to a file on disk.

(These are questionable design decisions, I know, but not the point of this post. Though feel free to comment on them, anyway).

I’ve written another program that reads the data file and graphs it. To get the most current data, I can send the USR1 signal to the daemon, which causes it to dump its data immediately. After restarting the renderer, I can analyze the most current data.

The tech (pregnant women and those faint of heart, be warned)

  • The daemon is written in TypeScript and executed through a on-the-fly transpiler in Node.
  • The data file is just a boring JSON dump.
  • systemd is in charge of starting and stopping the daemon
  • The renderer is a static web page served via a python3 server that uses compiled TypeScript to draw pretty lines on the screen via a charting library.
  • All runs on Linux. Mint, to be specific.

As I’m looking for general ideas for my problem, you are free to ignore the specifics of that tech stack and pretend everything was written in Rust.

Now to the question.

I would like to get rid of the manual sending of the signal and refreshing the web page. I would like your opinions on how to go about this. The aim is to start the web server serving the drawing code and have each data point appear as it is generated by the daemon.

My own idea (feel free to ignore)

My first intuition about this was to have the daemon send its data through a Unix pipe. Using a web server, I could then forward these messages through a WebSocket to the renderer frontend. However, it’s not guaranteed that the renderer will ever start, so a lot of messages would queue up in that pipe – if that is even possible; haven’t researched that yet.

I’d need a way for the web server to inform the daemon to start writing its data to a socket, and also a way to stop these messages. How do I do that?

I could include the web server that serves the renderer in the daemon process. That would eliminate the need for IPC. However, I’m not sure if that isn’t too much mixing of concerns. I would like to have the code that produces the data to be as small as possible, so I can be reasonably confident that it’s capable of running in the background for an extended period of time without crashing.

Another way would be to use signals like I did for the dumping of data. The web server could send, for instance, USR2 to make the daemon write its data to a pipe. But This scenario doesn’t scale well – what if I want to deliver different kinds of messages to the daemon? There are only so many signals.

  • MadhuGururajan@programming.devEnglish
    2·
    2 days ago

    might I suggest the following:

    1. Sensor Daemon and webserver can be in the same language/stack. Consider even making them the same program. your webserver needs to run always if you want users to be able to go to your site and get the output.
    2. Use a database like sqlite instead of a raw json dump as querying for data from your webserver becomes flexible (especially for charting purposes)
    3. Note: you can store json as a column in sqlite as well if you are concerned that you have too deep of nested data. just store them with keys that are required for your charts.
    4. you can achieve the circular buffer functionality naively by periodically checking number of rows > threshold and deleting the rows if it is exceeded.