Thursday, December 22, 2011

beaTunes 3.5 Early Access 3: Faster Startup, faster Data Fetching

beaTunes2 logoThis week's release is focusing on a number of issues with the internal database. As some of you may know, beaTunes uses a little embedded database called H2. It's open source, written in Java, and has served us well so far. In the previous 3.5EA versions I had switched the database engine to MVCC (Multi-Version Concurrency Control). As it turns out, this is not the best setting for the kind of workload beaTunes generates, as often times the entire database gets locked. Experiments have shown, that the MULTI_THREADED setting leads to far better results, as it allows to execute multiple SQL statements in parallel. This is particularly useful when executing a long running statement in the background, while still keeping the UI responsive. EA3 is now the first release with MULTI_THREADED enabled. As a result, I hope you see a more responsive, more fun application. For those of you with small libraries (less than 10,000 songs), the difference may not be noticeable at all.

While investigating optimal database settings, I also spent some quality time with certain key SQL statements in the application and was able to tune some of them, essentially removing a bunch of SQL joins. As a result, beaTunes should start up much faster, especially for those people with ten thousands of songs. Additionally, switching between different playlists should be a whole lot faster.

On the algorithm front not a whole lot has changed. Memory consumption for color calculation should be a little lower though, as I have rewritten some classes to have fewer or smaller members.

As always, there is some more info in the NOTES.txt file.

Alright, here's the obligatory warning: Before downloading and installing this, please make sure you understand what Early Access means:

  • Absolutely no warranty for whatever
  • Features may or may not work, appear, and disappear
  • It may not be possible to migrate data to future or previous versions (even though I make a reasonable effort)
  • This version will cease to function three weeks after its release
  • You cannot buy this version

And here are the download links:

Important

For those still using beaTunes 2: beaTunes 3.x changes beaTunes' internal database layout quite a bit, which takes a while. Do not interrupt this process, even if it takes very long! beaTunes 2.x will not be able to use the database anymore, after you ran beaTunes 3.x. If you intend to try 3.x and then go back to 2.x, make sure you back up the database before you try 3.x (depending on your OS it's in ~/Library/Application Support/beaTunes/Database, C:\Users\<username>\AppData\Local\tagtraum industries\beaTunes\database or C:\Documents and Settings\<username>\Local Settings\Application Data\tagtraum industries\beaTunes\database)! You also might want to check out this article about exporting beaTunes data.

Feedback

Please provide feedback! Let me know what does not work. Especially audio decoding problems, OutOfMemoryErrors and UI hangs and hiccups.

Thanks.

PS: The update from 3.0 to 3.5 will be free.

Labels:

Thursday, December 15, 2011

beaTunes 3.5 Early Access 2: Lowering Memory Consumption

beaTunes2 logoSince it's only been a week, there aren't too many changes. However, some of them are crucial, especially for those of you having huge collections. To make it short: EA2 contains a couple of fixes and a number of performance improvements.

As always, there is some more info in the NOTES.txt file.

Alright, here's the obligatory warning: Before downloading and installing this, please make sure you understand what Early Access means:

  • Absolutely no warranty for whatever
  • Features may or may not work, appear, and disappear
  • It may not be possible to migrate data to future or previous versions (even though I make a reasonable effort)
  • This version will cease to function one week after its release
  • You cannot buy this version

And here are the download links:

Important

For those still using beaTunes 2: beaTunes 3.x changes beaTunes' internal database layout quite a bit, which takes a while. Do not interrupt this process, even if it takes very long! beaTunes 2.x will not be able to use the database anymore, after you ran beaTunes 3.x. If you intend to try 3.x and then go back to 2.x, make sure you back up the database before you try 3.x (depending on your OS it's in ~/Library/Application Support/beaTunes/Database, C:\Users\<username>\AppData\Local\tagtraum industries\beaTunes\database or C:\Documents and Settings\<username>\Local Settings\Application Data\tagtraum industries\beaTunes\database)! You also might want to check out this article about exporting beaTunes data.

Feedback

Please provide feedback! Let me know what does not work. Especially audio decoding problems, OutOfMemoryErrors and UI hangs and hiccups.

Thanks.

PS: The update from 3.0 to 3.5 will be free.

Labels:

Friday, December 9, 2011

beaTunes 3.5 Early Access 1: Faster, more responsive

beaTunes2 logoSurprising to some, I have not been idle in the past couple of months since the first release of beaTunes 3. In preparation of beaTunes 3.5, I have spent a lot of time on improving Jipes and the SampledSP collection. Both are open source libraries and central to beaTunes 3.x. Let me explain why..

Traditionally, Java offers very poor multimedia support. To illustrate this: In Java you can't just load and play an AAC file. Instead you have to find or write some code that does it for you. Support out of the box is not available. The same is true for all but the most simple (PCM) or obscure (μ-law) encodings.

The SampledSP collection remedies this situation by providing a bridge to native audio libraries like Apple QuickTime (32bit), Apple CoreAudio (OS X) and Microsoft MediaFoundation (>= Windows 7). Furthermore, it provides basic resampling capabilities.

beaTunes 3.5 will take full advantage of these libraries. It will also better take advantage of Jipes. Here's how: To analyze, all versions of beaTunes prior 3.5 first decoded the audio file to memory. Then, one after another, each requested algorithm would operate on the decoded audio data. While this approach is very efficient for multipass algorithms (decode once, process many times), it is flawed, when you want to process many files in parallel, as decoded audio can easily be 50MB and bigger. And once you have used lots of memory, the system becomes less responsive, because the OS is busy with memory management... not a pleasant situation. Starting with version 3.5, beaTunes processes small chunks of audio at a time, drastically reducing peak memory consumption, thus easing the strain on the OS. At the same time, processing steps that many algorithms have in common are only executed once, thus saving some work. The magical framework that makes this possible is Jipes.

To make this all work in beaTunes 3.5, I had to rewrite some of the existing algorithms and the task handling infrastructure. Most plugin APIs stayed unchanged, though. Additionally, the bundled JRuby and Groovy scripting engines were updated to current versions.

Better Responsiveness

While the audio processing stuff is probably the biggest change under the hood, I have also spent some time on other, more immediate performance aspects. One of them being UI responsiveness. When playing with 3.5, you will notice that scrolling around in playlists is a lot smoother than it used to be. There may be a slight flicker when beaTunes has to load data from the database, but typically values should appear in less than 0.5s. In fact, I have not seen this take more than 0.25s. Another area of work has been general responsiveness during standard tasks, like adding analysis tasks to the queue. Here, you should also notice an improvement.

Last but not least, I rewrote much of the online resources interaction code. Before 3.5, beaTunes first sent all data to the server, processed all the results and then added the unsuccessful tasks to the queue for local processing. Considering, that server processing takes a long time, but not a whole lot of resources, while local processing takes a lot of resources and a lot of time, this is a poor approach. In version 3.5 beaTunes sends data to the server (indicated by a message in the status bar of the task queue). In parallel, all tasks are added to the queue. But unlike before, beaTunes does not wait for all results from the online database to be processed before processing the remaining tasks locally. Instead, a task in the queue is processed locally right after we received data from the server for the task at hand (if at that point it is still necessary at all). In essence, both ways of processing (with online resources and locally) are more or less parallelized, thus significantly reducing the total processing time.

One last but important remark about colors: beaTunes 3.5 calculates colors slightly differently. If you have a big collection (i.e. more than 10,000 songs), this can slow down analysis significantly, as many colors (not just for the song you added to the queue) have to be changed in the background. Also, colors calculated with prior versions will have a different meaning, thus similarity can only be computed correctly, after all colors have been re-calculated.

As always, there is some more info in the NOTES.txt file.

Alright, here's the obligatory warning: Before downloading and installing this, please make sure you understand what Early Access means:

  • Absolutely no warranty for whatever
  • Features may or may not work, appear, and disappear
  • It may not be possible to migrate data to future or previous versions (even though I make a reasonable effort)
  • This version will cease to function one week after its release
  • You cannot buy this version

And here are the download links:

Important

For those still using beaTunes 2: beaTunes 3.x changes beaTunes' internal database layout quite a bit, which takes a while. Do not interrupt this process, even if it takes very long! beaTunes 2.x will not be able to use the database anymore, after you ran beaTunes 3.x. If you intend to try 3.x and then go back to 2.x, make sure you back up the database before you try 3.x (depending on your OS it's in ~/Library/Application Support/beaTunes/Database, C:\Users\<username>\AppData\Local\tagtraum industries\beaTunes\database or C:\Documents and Settings\<username>\Local Settings\Application Data\tagtraum industries\beaTunes\database)! You also might want to check out this article about exporting beaTunes data.

Feedback

Please provide feedback! Let me know what does not work. Especially audio decoding problems, OutOfMemoryErrors and UI hangs and hiccups.

Thanks.

PS: The update from 3.0 to 3.5 will be free.

Labels: , , , ,

Wednesday, December 7, 2011

Just released: beaTunes 3.0.11

beaTunes2 logoI'm currently in the process of preparing the first (and potentially only) Early Access release of beaTunes 3.5. In the process of working on the new version, a couple of small fixes appeared that I thought should make it into the current beaTunes 3 branch. None of them is earth shattering, but it all adds up. The brand-new beaTunes 3.0.11 contains these fixes. You might want to update.

Download