Blog Status : Moved, not dead!

Hi all,

Once upon a time I posted on this blog (somewhat) regularly. Currently I dont. Why? I'm running OpenAV Productions, and that is where the updates are!

If you're still interested in Linux audio, C++ programming or software in general, checkout the site:
www.openavproductions.com

Developers may have particular interest in the developer topics like implementing NSM or dealing with memory in real-time.

Audio programming folks, checkout some articles that I've written on the topics of real-time programming, memory management, implementing NSM and more:
http://openavproductions.com/conferences.

I probably won't post here for another long time, so bye for now! -Harry

Real time audio programming languages

Introduction

Over the last couple of years I've written various real-time audio programs. Its difficult to adhere to real-time regulations: you've got to get threading and memory management right.

C++ is the often concidered the obvious choice language for large real-time audio programs: its a compiled language, and is deterministic in time if used carefully. This is necessary for real time (RT) work, and rules out VM based languages like Python for any low-latency work.

In C++ there's many a way to achieve real-time, one of which I have detailed here: https://github.com/harryhaaren/fypRealtimeCppPrograming

Other languages

C++ is one way to go, but in recent times there are various other programming languages which are becoming increasingly attractive to the real-time audio programmer. Particularly these two languages have caught my eye recently:

Rust : http://www.rust-lang.org
Iolanguage: http://iolanguage.org

Both of these languages have certain characteristics which make them possible candidates for RT programming.

Rust

Language Overview

Rust is a language that focuses on "blocks", using boundaries. Integrity, availability and concurrency are its main goals. It uses lightweight tasks with message passing for concurrency, no shared memory.

The Interesting Stuff

I'm most intrigued by the memory management of the language: everything is static unless declared "mut" (or "mutable"), and ownership of objects is very strict. This means that managing resources in a real-time safe way is well defined, and hence the code will be maintainable.

Three different "pointer" types exist, as well as new concepts like owned boxes and managed boxes... these new concepts may aid memory allocation troubles, but perhaps it complicates them too, I don't have much experience yet with it, so only time will tell...

Learning It

Most of what I know comes straight from their homepage or tutorial:
Homepage: www.rust-lang.org
Tutorial: http://static.rust-lang.org/doc/0.6/tutorial.html

 

Conclusion

A cool language, and if the memory concepts prove useful, it could be an awesome new language to learn for the audio-programming enthusiast.

 

IOlanguage

Language Overview

This is a smalltalk inspired language, while also incorporating various different elements from other languages together. Actors based concurrency is used (a la Act1), while it is also kept small for embeddable purposes. Runs in a small VM.

The Interesting Stuff

Intensive inspecting of object instances / program state (like LISP) aids debugging significantly. Extensive concurrency possibilities: co-routines, actors, futures and yield statements allow for flexible "time" programming.

 

Learning It

Extensive documentation and example code here:
http://iolanguage.org/scm/io/docs/IoGuide.html#Introduction

Conclusion

Cool language, unfortunately probably not fully real-time safe / deterministic due to running in a VM.

Sum Up

"So what language will I use for my next project?" I hear you ask: well I'm staying with the tried and tested C++ for a while. I've dabbled with Vala previously ( see ValaLooper and Prehear ), but they're not quite suitable to RT work in my opinion.

Although its nice to work with a slightly higher level language, its hard to determine if the generated code is genuinely real-time safe.

The perfect real-time safe code for me is code that is so simple, that proving its real-time safe under any conditions is trivial. Then the code is maintainable and readable.

Know of any RT capable language I've left out? Get in touch: I'm interested in hearing about it!

LV2 and Atom communication

EDIT: There are now better resources to learn LV2 Atom programming: please use them!
www.lv2plug.in/book
http://lac.linuxaudio.org/2014/video.php?id=24
/EDIT


Situation: You're trying to write a synth or effect, and you need to communicate between your UI and the DSP parts of the plugin, and MIDI doesn't cut it: enter Atom events. I found them difficult to get to grips with, and hope that this guide eases the process of using them to achieve communication.

 

Starting out

I advise you to first read this : http://lv2plug.in/ns/ext/atom/
It is the official documentation on the Atom spec. Just read the
description. It gives a good general overview of these things called Atoms.

This is "message passing": we send an Atom event from the UI to the DSP part of the plugin. This message needs to be safe to use in a real-time context.

(Note it is assumed that the concepts of URIDs is familiar to you. If they're not, go back and read this article: http://harryhaaren.blogspot.ie/2012/06/writing-lv2-plugins-lv2-overview.html )

Step 1: Set up an LV2_Atom_Forge. The lv2_atom_forge_*   functions are how you build these events.

LV2_Atom_Forge forge;
lv2_atom_forge_init( &forge, map ); // map = LV2_URID_Map feature

Atoms

Atoms are "plain old data" or POD. They're a sequence of bytes written in a contiguous part of memory. Moving them around is possible with a single memcpy() call.

 

Writing Atoms

Understanding the URID naming convention

// we need URID's to represent functionality: There's a naming scheme here, and its *essential* to understand it. Say the functionality we want to represent is a name of a Cat (similar to the official atom example). Here eg_Cat represents the "noun" or "item" we are sending an Atom about. eg_name represents something about the eg_Cat.

something_Something represents an noun or item, while something_something (note the missing capital letter) is represents an aspect of the noun.

LV2_URID eg_Cat;
LV2_URID eg_name; 

In short classes and types are Capitalized, and nothing else is.

Code to write messages

// A frame is essentially a "holder" for data. So we put our event into a LV2_Atom_Forge_Frame. These frames allow the "appending" or adding in of data.
LV2_Atom_Forge_Frame frame;


// Here we write a "blank" atom, which contains nothing (yet). We're going to fill that blank in with some data in a minute. A blank is a dictionary of key:value pairs. The property_head is the key, and the value comes after that. Note that the last parameter to this function represents the noun or type of item the Atom is about.
LV2_Atom* msg = (LV2_Atom*)lv2_atom_forge_blank(
                &forge, &frame, 1, uris.eg_Cat );

// then we write a "property_head": this uses a URID to describe the next bit of data coming up, which will form the value of the key:value dictionary pair.
lv2_atom_forge_property_head(&forge, uris.eg_name, 0);
 
// now we write the data, note the call to forge_string(), we're writing string data here! There's a forge_int() forge_float() etc too!
lv2_atom_forge_string(&forge, "nameOfCat", strlen("nameOfCat") );

// Popping the frame is like a closing } of a function. Its a finished event, there's nothing more to write into it.
lv2_atom_forge_pop( &forge, &frame);

 

From the UI

// To write messages, we set up a buffer:
uint8_t obj_buf[1024];

// Then we tell the forge to use that buffer
lv2_atom_forge_set_buffer(&forge, obj_buf, 1024);

// now check the "Code to write messages" heading above, that code goes here, where you write the event.

// We have a write_function (from the instantiate() call) and a controller. These are used to send Atoms back. Note that the type of event is atom_eventTransfer: This means the host should pass it directly the the input port of the plugin, and not interpret it. write_function(controller, CONTROL_PORT_NUMBER,
               lv2_atom_total_size(msg),
               uris.atom_eventTransfer, msg);

From the DSP

// Set up forge to write directly to notify output port. This means that when we create an Atom in the DSP part, we don't allocate memory, we write the Atom directly into the notify port.

const uint32_t notify_capacity = self->notify_port->atom.size;
lv2_atom_forge_set_buffer(&self->forge,
                         (uint8_t*)self->notify_port,
                          notify_capacity);
 
// Start a sequence in the notify output port
lv2_atom_forge_sequence_head(&self->forge,
                             &self->notify_frame, 0);

Now look back at the "Code to write messages" section. that's it, write the event into the Notify atom port, and done.

Reading Atoms

// Read incoming events directly from control_port, the Atom input port
LV2_ATOM_SEQUENCE_FOREACH(self->control_port, ev)
{
  // check if the type of the Atom is eg_Cat
  if (ev->body.type == self->uris.eg_Cat)
  {
    // get the object representing the rest of the data
    const LV2_Atom_Object* obj = (LV2_Atom_Object*)&ev->body;
 
    // check if the type of the data is eg_name
    if ( obj->body.otype == self->uris.eg_name )
    {
      // get the data from the body
      const LV2_Atom_Object* body = NULL;
      lv2_atom_object_get(obj, self->uris.eg_name,
                          &body, 0);
      
      // convert it to the type it is, and use it
      string s = (char*)LV2_ATOM_BODY(body);
      cout << "Cat's name property is " << s << endl;
    }
  }
}

Conclusion

That's it. Its not hard. It just takes getting used to. Its actually a very powerful and easy way of designing a program / plugin, as it *demands* separation between the threads, which is a really good thing.

Questions or comments, let me know :) -Harry

MlTutorial: Working with the MediaLovinToolkit

Hi!

I've been interested in doing some video coding for a while now, but never really got into it yet. Until today, when I re-attempted (yes, I'd tried before :) to achieve some simple functionality with MLT.

Initially I found it very difficult to find any resources as to how one can use the MLT framework from C++, but some googling led me to various resources scattered around the internet.

The MLT github repo as a super-simple example (which although informative doesn't scale up to the use of filters or any advanced functionality):
https://github.com/mltframework/mlt/blob/master/src/examples/play.cpp

A search around the net showed me this post on a forum http://ubuntuforums.org/showthread.php?p=7370184
This seemed to be more along the lines of what I had hoped for, however the code segfaults upon running...

Finally the "tests" subdir in the MLT tarball provide some test program code, but its difficult to understand (IMO) as its not commented for learning purposes: https://github.com/mltframework/mlt/tree/master/src/tests

So between these resources I've decided to bunch together some examples of how to use MLT using C++. The code is online on github, and may be useful to others hoping to learn to use the MLT framework.

There's currently two "playback" tutorials, and one "filter" tutorial. Reading them will show the rough design of the MLT library, and how to use it. Advanced functionality tutorials will be added as I learn it myself :)
https://github.com/harryhaaren/mltutorial

Welcoming issues / merge requests from MLT users / devs / anybody!
Cheers, -Harry

Getting to grips with Ganv

So you want to create a "Graph" based program? Use Ganv! It looks awesome, and David Robillard has already done the hard parts :)

Starting Out:

We're just doing a single file, proof of concept right now. Nothing fancy yet.
So you'll need to install:
ganv
gtkmm

Easiest way to do this is just checkout Drobilla's svn:
svn co http://svn.drobilla.net/lad/trunk drobillad

./waf configure --prefix=/usr
./waf
./waf install

What next:

Install ganv system wide. Download this tutorial's code from here: https://github.com/harryhaaren/openAudioProgrammingTutorials/blob/master/flowCanvas/flowcanvas.cpp

Read the code, its pretty self explanatory.

Then what?

Its a good idea to get to grips with how all this works together:

Ganv: is the way you interface with the whole
FlowCanvas: is the canvas itself, behind the scenes.
GTKmm: is your window, and you do what you like with it.

So the final structure is somewhat like this:
Gtk window
     Ganv Canvas
            Ganv Module
                  Ganv Port
                          Ganv Edge (connects ports)

That's all for now, hope I saved you a bit of time and effort! -Harry

Luppp: Status as of Nov '12?

Hey everybody,

I'm writing to let you all know that the Luppp project is not dead. I've not posted here about Luppp since May, and also haven't pushed code to the repo at github for 6 months. So what is the status?

Luppp is "taking the back seat" for a while. I mean that real-life has currently taken over, and with my final year of college and projects on, I don't see myself spending a lot of time on Luppp before the next LAC (that's the middle of May).

Don't worry, the reason for not pushing code recently is only partly due to time, the other reason is that during the summer I started doing (another...) total re-write. This time I'm confident that the code is a lot more maintainable. I hope :)

Till next, -Harry

Writing Lv2 GUI's: Making it look snazzy

So we know how to build a basic "amp" plugin, but there's no GUI. We like shiny gui's right? Of course we do :) This tutorial will explain how to make one.

Gooeys - Or GUI's:

The GUI part of a plugin can be viewed as a totally separate entity to the "DSP" or audio part of the plugin. We will call these the "halves" of our plugin. Theres a little bit of theory to cover before we start looking at code. The next communication section describes how the DSP and GUI parts of the plugin can inform eachother about whats going on.

Communication:

So the GUI runs in the "GUI" part of the host program, and the DSP runs in the audio part of the host program. We can't communicate directly between the halves due to details that don't really concern us. (threading and thread safety if you must know..)

How it works: The two parts of our plugin are loaded into the two parts of the host program. How we communicate between the two "halves" of our plugin is by using Lv2 ports. These ports send communicate data between the host and the plugin. The host can then use that same data to update the GUI.

Similarly the GUI can write data, which will then be sent to the audio half by the host. That's all there is to it!

Choosing a toolkit

So there's a lot of GUI toolkits out there. The most prominent open-source ones are GTK and QT. This tutorial will explain how to create a custom GTK interface for an Lv2 plugin. My reason for choosing GTK? Its what I know. I also like it. If somebody would contribute code for doing the same in QT, I will gladly link to it from here.

Please note that I'm using the C++ wrapper around GTK (called GTKmm), and I use version 2, not the new stable GTK 3. I know that GTKmm 2 works, and have no reason to update to GTKmm 3.

Side note (skip if you want to get hands-on GUI writing): The loading of GUI plugins into hosts is done using the SUIL library. It is useful to know that it has certain toolkits that it supports, currently X11, GTK2 and QT. If you want to use a different toolkit please read the SUIL library page.

Making a GUI

So what do we actually do to get a GUI for an Lv2? We write a widget. If you're not at all familiar with GUI programming, I will suggest you look at the GTKmm2 Drawing Area tutorial code: Drawing a custom widget using a GTK::DrawingArea. When we have a widget created, we need to pass that widget to the Lv2 host, which will display it for us.

Code

Yes. Where is it? There is a copy of the Lv2 repo on my Github, where I will publish the code for this tutorial. Chances are that these tutorials will be merged into the main Lv2 examples from the Github page.

Communication in code

So we talked about the theory of communication between the halves, but how do we actually do that in code? The Lv2 UI extension that we use to load the GUI gives us two "things" that we need to write port events from the GUI to the host. What these things represent is not really important: we just need to use them. (if you must know: LV2UI_Controller is a pointer to the host's own stuff, and LV2UI_Write_Function is a function pointer that will do the work)

Github Lv2 repo: SinSynth folder at commit while writing this tutorial

Writing port events in the GUI

So we have these two things: LV2UI_Controller and LV2UI_Write_Function, how do we use them? More theory first. Events happen in the Widget, ie: if the user clicks, our Widget gets a function called. That function must decide what to do, and then call the write_function().

What this means in practical terms, is that the Widget class needs access to the LV2UI_Controller and LV2UI_Write_Function.

The way that I solve that, is by adding the LV2UI_Controller and LV2UI_Write_Function to the Widget class. Some people might say this is ugly design: but its simple and works well.

Look at Widget::on_button_press_event to see the code in action.

 

Testing the plugin

There's a program called Jalv, which is a great little host to test out your Lv2 plugins with: I've used it to develop this tutorial and the GUI for the tutorial. Grab it here: http://drobilla.net/software/jalv/


Running jalv.gtk http://lv2plug.in/plugins/eg-sinsynth
gives me this screenshot, clicking it will change the frequency!

 

That's all!

You've now made a custom GUI, that should work as a Lv2 plugin gui. Sure its not UI artwork yet, but I'll leave you to explore Cairo and its cool graphics functions on your own for a while ;)