I’ve come to the difficult decision that I need to archive the logstash-output-jdbc project.
I apologise to anyone depending on it. Speaking frankly it should not be a surprise as it’s effectively been unloved since Logstash 2.0 and I have been asking for others to come forward and assist for sometime.
logstash-output-jdbc is a very small project (and quickly hacked together project) that is easy to understand and is honestly ripe for either rewriting or refactoring with the new features Logstash has implemented over the years. If you feel you can take over the project please reach out, I’ll unarchive the project and make you a maintainer, or I will link your fork in the project.
Sadly, I no longer use this at work and I simply cannot dedicate the personal time required to provide support and keep up with Logstash itself. Over the last few years I’ve worked sporadically to upgrade, fix issues and provide support. I’ve realised that is not fair to the users of the plugin, nor myself. So I’m calling it.
Reflecting on the project I feel I made some mistakes. Many of them are not new to open source maintainers of any scale and I feel foolish I had not stepped back and thought about it critically sooner.
- I felt like the project was too small/a quick hack job and didn’t need the love and attention it really required.
- What started off as a quick 2 hour project after work to solve a very specific need at work should probably have stopped there.
- I should have been honest with myself about the amount of time I was willing to spend and budget accordingly once I no longer used the project at work.
- Although I knew that they existed I completely under-estimated small details between different SQL engines that weren’t papered over by JDBC. Early in the project people asked for samples for various SQL engines and I put time into making sure to find workarounds and maintain backwards compatibility in configuration, etc. As I added examples I found myself spending more time finding workarounds.
- I should have setup benchmarking as part of the test suite from the outset.
- I should not have offered to support systems I do not use/know well, for free (often which resulted in basic issues relating to the environment or the user trying to bend Logstash to their will and use it other than was intended).
- I should have found a way for corporate users to fund the project. Although there wasn’t a huge amount of support often it was relatively involved and came in via email. Of those Oracle users often needed the most support and in some cases would’ve happily paid for my time, if only there was something other than paypal/flattr/etc.
- I should have been more frank when I felt like the user was trying to turn Logstash and logstash-output-jdbc into a general purpose ETL system, where other tools like Mulesoft, Pentaho kettle, etc. are probably better suited. This happened often enough that I am surprised that Logstash hasn’t already considered going down this route.