The Kdenlive Conundrum

[GMaq]

Hi,

I'm sure some folks are beginning to wonder if AV Linux 6.0 is ever going to be released, and don't worry it is... however between my job and situations like I'm about to describe it has been delayed a bit...

Kdenlive is a great and ambitious project and for many good reasons it is probably the best known Linux Video Editor. Unfortunately Kdenlive is a juggle of ffmpeg, MLT and Kdenlive (which is essentially the User Interface) and because it has to follow ffmpeg which has undergone MAJOR forks and changes in the past year it has become increasingly dependent on cutting edge development to function properly. Anyone who follows the project will know that the only way to use it effectively is to rely on GIT builds between releases because curiously the last few official 'releases' have been riddled with showstopping bugs. Kdenlive Ubuntu users are encouraged to to use a 'kdenlive-svn' PPA because the official Ubuntu packages are usually broken. Anyone following AV Linux will also know that the last few AV Linux releases had custom GIT builds of both Kdenlive and MLT to function properly.

As an end-user this is mildly inconvenient, as a packager/distributor this is maddeningly time consuming and frustrating to have to watch a project so closely and build and re-build packages around every official release. AV Linux is currently remaining with ffmpeg more or less in it's original form which is ffmpeg used in conjunction with libav packages, this has given us a very solid and reliable solution across many ffmpeg dependent apps including Kdenlive until now. MLT which is the underlying framework used by both Kdenlive and Openshot has transitioned to working better with the newer 0.8.X branch of ffmpeg than with the 0.7.X branch that AV Linux uses, the issue is that the way Debian and the 3rd party 'deb-multimedia' handle packaging ffmpeg is vastly different due to a long-running difference of opinion and AV Linux utilizes 'deb-multimedia' for ffmpeg and many other related packages that have worked well for a long time, suffice it to say that changing a working core library like ffmpeg because it only affects ONE outside library (MLT) is not something I am willing to do at this time because of the vast amounts of collateral damage it may cause to other applications.

How this plays out is... The current Kdenlive (0.9.2) requires at least MLT 0.7.8 which has a known bug in it which causes rendering to crash (the Kdenlive forums and Mantis bug catcher will back this up). The reasons for this were not terribly clear but there was some discussion that it might be related to certain versions of ffmpeg so a new version of MLT 0.8.0 was created which had a bug in it that caused it to not build on Debian Squeeze/AV Linux: http://kdenlive.org/forum/solved-mlt-080-fails-see-qt4-qimage-build-libs-when-present and http://kdenlive.org/forum/more-script-failure-solved. This bug was fixed in MLT's GIT and it now MLT 0.8.1 can be built with AV Linux however with ffmpeg 0.7.X the rendering crashes still occur and in addition MLT 0.8.0+ breaks the library soname links with Openshot Video Editor so it no longer works... This is a major problem because AV Linux is the official LiveDVD endorsed by the Openshot project!

Sooo...

Despite MANY hours of research, reading and packaging it is not possible to have a functional current version of Kdenlive on AV Linux 6.0... I don't blame anyone, I'm not a developer at that level so who am I to question what decisions they make. I will say the fact that a project as popular as Kdenlive requires a special SVN PPA with 'between release' versions simply to function on Ubuntu is an intriguing way to operate, I sincerely hope that their upcoming refactoring will make Kdenlive a bit more accessible to your average Linux user.

Kdenlive also has a series of build scripts to build a sandbox version with all the required dependencies in your home folder, this is not a way I would ever consider including it for distribution in AV Linux however for folks desperate to have Kdenlive it will be the only avenue for Kdenlive on AV Linux 6.0, please note that the build scripts are quite often broken by new developments as well but are maintained as needed.

This has not been an easy or happy decision to have to make but shipping AV Linux with a hobbled version of Kdenlive does neither the Kdenlive devs nor myself much good in the long run.  

[davephillips]

Greetings,

A note to anyone who might want to try the build script: Go for it ! I use the script to generate a usable Kdenlive on each of the systems running here. As G points out, it's a good editor but the official releases can be rather frustrating. I'm putting it mildly.

However, the binary generated by the script works very well. It downloads current sources for everything required, so it takes a while to build, and again as G mentions, sometimes the script gets corrupted or the build is subject to problems introduced in the external source files. Subsequent builds are kept in their own directories, so if you build a working version and the next one craps out, you can always return to the earlier-dated directory.

Just two pesetas from another user wrestling with Kdenlive.

Best,

dp

[trulan]

All I can say is - wow.  I have been recommending kdenlive to my friends as a great free video editor, and none of them were able to get it working outside of AVLinux.  Now I understand why.  I'd hate to have to be without it though.

I like the idea of a 'sandboxed' kdenlive though, and now that I think about it I believe I did use the build scripts a while ago.  Guess we'll need a 'how-to', huh?

[GMaq]

Guess we'll need a 'how-to', huh?

That's an excellent idea! Any volunteers? I've never used the scripts personally and I've got my hands full right now with the User Manual and some last minute stuff. But a clearly detailed guide to using the build scripts would certainly mitigate some of the inconvenience of this whole issue.

[jade_1_4]

All I can say is - wow.  I have been recommending kdenlive to my friends as a great free video editor, and none of them were able to get it working outside of AVLinux.  Now I understand why.  I'd hate to have to be without it though.

Hi everyone,
That has been my experience as well. At first this news shocked me Kdenlive, quirky and capricious as it might be, is the closest OSS has got to a full featured user-friendly non linear video editor. The fact that a stable version of Kdenlive came pre-installed in AVLinux separated it from other AV production distros such as ArtistX, KXstudio, dyne:bolic, DreamStudio, UbuntuStudio, etc.. or even all-included generalist distros such as Knoppix. This allowed my photo students and OSS workshop participants to skip installation (running from USB-key in borrowed computers), codec/dependency hell, compiling and jump straight to video editing and AV narrative structuring, as in this video http://vimeo.com/40746584#at=0

I am very grateful to Gmaq and this community for their amazing work producing and constantly improving the most reliable, no updates no hassles, standalone artists distro Ive had the pleasure of using. I just donated 25$ (might not sound like much but I make my living as a photographer and teacher in Central America..) and pledge other 25$ to the AVLinux project if someone writes a clear working guide to using the mentioned build scripts for a "sandboxed kdenlive" in AVLinux 6.0 as indeed, it seems no easy task, as per StudioDave description: http://www.linuxmusicians.com/viewtopic.php?f=47&t=8800at=0

[davephillips]

I... pledge other 25$ to the AVLinux project if someone writes a clear working guide to using the mentioned build scripts for a "sandboxed kdenlive" in AVLinux 6.0 as indeed, it seems no easy task

It works like this:

You acquire the Kdenlive build script from http://www.mltframework.org/twiki/bin/view/MLT/BuildScripts. Put it in your personal home directory (e.g. /home/dave) and leave it for the moment.

You will NOT need to install the Kdenlive dependencies, that's what the script does. It will build Kdenlive in its own directory (e.g. /home/dave/kdenlive/20120713), don't worry about that. You WILL need various retrieval and build tools, so make sure you have the tools and utilities descrived here:

http://www.kdenlive.org/user-manual/downloading-and-installing-kdenlive/installing-source/installing-required-libraries

See the requirements under General Tools and General Libraries. Btw, if you're compiling for a 64-bit system you must use yasm, not nasm. If you have both on your system get rid of nasm, you won't need it anyway if you have yasm. (It's a piece of software called an assembler, btw).

If you're certain that you have the necessary build tools then you neeed only launch the script from its directory (/home/dave on my box). Open an xterm, change to the directory where the build script is located and run it so :

cd /home/dave
sh build-kdenlive.sh

Once it starts it automatically retrieves and builds the required dependencies, and finally builds Kdenlive. If all goes well the build will end with a Success message. At that point you can start the program from the dated directory :

cd /home/dave/kdenlive/20120713
./start-kdenlive

If you've been having trouble with slow performance run it with the "--graphicssystem raster" option (minus the quotes) :

./start-kdenlive --graphicssystem raster

I put this command in shell script named "go" so all I do to start the program is run ./go in the dated directory. On my Arch and AVLinux systems I add a launcher to my desktop, and voila, I have a 1-click solution to running a stable version of Kdenlive.

The script isn't magic. If the upstream devs have screwed up something in the repos then it'll screw things up during the script execution and the build will fail. You'll know why, though. (There will be error messages in plenty). And occasionally one of the Kdenlive devs will change a setting that will bork the script.

It's all so much fun !!!

Not. But hopefully this little guide might inspire someone to try the script too. It isn't painful unless something goes wrong. Which, given that this is Kdenlive and Linux ... Let's just hope for the best.

Now, if this works for you, someone owes G some $$$ ...

Finally, a little advertisement. I wrote an article about building audio software on Linux, it might be of some assistance to the bewildered :

http://www.linuxjournal.com/content/building-linux-audio-applications-101-users-guide-part-1

And here's one on troubleshooting Linux audio problems :

http://www.linuxjournal.com/node/1000244

Might be helpful to someone.

Best,

dp

"Bless all forms of intelligence."

[GMaq]

Hi,

Thanks very much Dave! If we need to go that route this is invaluable info! All the build dependencies for Kdenlive are always included in AV Linux so unless the scripts are broken everything needed should be in place.

As a plan 'B' I've approached the Kdenlive developers on bended knee: http://kdenlive.org/forum/sincere-noteplea-developers-kdenlive

I will wait a few days and see if I get a response, beyond that no promises, but there might be a remote possibility of getting a sandboxed version to run in /opt which is where I stow other bundled binary stuff like Blender etc. but obviously the best case scenario would be regular packaging.

[RickBrock]

I don;t understand...

what is a 'sandbox version'?

what is 'svn'?

what is 'squeeze'?

what is 'MLT'?

these, I think, are things i need to start understanding... please help

thank you.
Rick

[Prof Knaakenbroed]

Hi Rick,

let me help you.

http://en.wikipedia.org/wiki/Sandbox_%28computer_security%29

http://en.wikipedia.org/wiki/Apache_Subversion

http://wiki.debian.org/DebianReleases

http://www.mltframework.org/bin/view/MLT

Yours
ProfKnaakenbroed

[GMaq]

Ok....

Some progress, I have been experimenting with the build script, it is currently broken unless I disable building the frei0r plugins, however I was able to get it up and running by manually copying over the frei0r plugins already installed on the system. So yay! I have a sandboxed Kdenlive 0.9.2 up and running in my home folder and a couple of test renders did not crash! Great news for me but not distributable yet for everyone else. I have renamed and copied the folder into /opt and have a lot of file permissions issues to figure out to get it to run properly and keep it's configs.

This is a messy way to get what we want but unless I hear from the Kdenlive devs about fixing the libs for normal packages it may be a way to keep Kdenlive in the mix. Next step is to test a LiveDVD and see what happens but it's 3:10AM right now so I think I'll tackle that another day!

[trulan]

@RickBrock:  This stuff is programmer's lingo.  As a 'normal' user, you should be able to get along quite well without knowing all that stuff.  Nevertheles, if you want to learn about it, by all means do!

I'll try to explain them in plain English.
1. MLT and ffmpeg: these are software libraries that are used by various programs.  MLT helps a program to work with video, ffmpeg helps it to read, create, and convert, many different types of audio and video files.  These are normally shared among all the programs on your system.  KDEnLive and OpenShot share MLT, many programs share ffmpeg.  Almost all Linux programs use such shared libraries (also called dependencies).  This helps keep the system smaller, and is supposed to help avoid bugs, because if you fix a bug in a shared library, it will be fixed for every program that uses that library.  Sometimes, though, this seems to cause more trouble than it saves...

2. Sandbox version: a program that is sandboxed is restricted to running in its own area.  It does not use libraries or files from the rest of the system, instead it includes its own, so everything is self-contained and will not affect anything else on your system.  This takes up a little more space, but it allows you to do things such as use a different version of MLT and ffmpeg for KDEnLive than for OpenShot (thus working around the problem GMaq is currently having).

3. SVN (SubVersioN): a programmer's tool to track changes they make in a program.  Think of it as little miniature versions between released versions.

4. Squeeze: the code name for Debian Version 6.0.  All Debian versions are named after characters from the movie Toy Story.  Hence we have Lenny (older), Squeeze (current version), Wheezy (testing version, will be Debian 7.0), and Sid (unstable development, named after the kid who broke all the toys in the movie).

[trulan]

@GMaq:  Hmm, I just finished running the kdenlive build script and it completed with no errors.  Not sure if the problem you had with the frei0r plugins was related to something in Squeeze (you'll recall I have a pretty mixed/mixed-up system), or if it was fixed during the night - but it all works here.  Now to try moving it all to /opt ....

Oh, btw, 'About kdenlive' reports:
Version 0.9.3 (rev. 9f8da3a)
Using KDE Development Platform 4.6.5 (4.6.5)

Edit:  OK, got it running in /opt.  There were some files in the .kde directory that pcmanfm couldn't copy, but a 'cp -r' did the job and now it works.  So far all the kdenlive files in /opt are still read-only except for root.  It seems to work fine.

[jade_1_4]

Thanks a lot Dave, GMaq and trulan, this is what I call community and one of the main reasons I prefer Linux. You people are amazing. I-ll wait for the 6.0 release  to check how the build script works for this all-thumbs no-brain user - 20 years of toxic fumes in the darkroom take a toll ya know !!

Wont this build testing change once the final release is out ? Am I right to think build scripts are vulnerable to the changing state of dependencies, basically making them developer-oriented solutions ? I-ll give it a try and post my experience here, but it would be plain AMAZING if AV Linux 6.0 came with the "sanboxed version" in /opt for all us, Terminally challenged users 

Once I get it working I-ll be sure to put my money where my mouth is and donate the additional 25$ as pledged - and I have never paid before for a piece of software other than the "MicroSoft Tax", that important is a rock-solid Kdenlive to me !!

[GMaq]

@jade_1_4

First thanks very much for your support,  you can see the amount of time that can go into getting one single program running and Kdenlive isn't the only problematic one by far.

My objective at this point is to get a properly configure and fully functional Kdenlive build on to AV Linux 6.0, you are correct that at this point it will be up on the user after an install to re-run the build script for updates (unless the Kdenlive devs make some changes). Once I get a working version installed in /opt I am going to make a deb package of it so it can be removed in order for people to continue on with the build script in their own respective home folders. Also if there are problems people can always re-install my deb to get back to a working version.

In my opinion something as volatile as Kdenlive should just be left alone if it's working anyway!

@trulan,

My guess is you have newer versions of libcv-dev and libgavl-dev, these are no longer upgradeable from Wheezy without totally upsetting the applecart, If I run ./configure on the frei0r source folder that the build script downloads it all checks out dependency-wise but the build fails. Perhaps davephillips or someone else who uses the build script regularly can comment, but I doubt anyone fixed the script, I posted about this issue on the Kdenlive forum and there has been no reply yet.

http://kdenlive.org/forum/build-script-fails-frei0r-plugins

[RickBrock]

@Trulan and ProfK
Thanks so much, that is very helpful/

@Gmaq - star!

[GMaq]

OK...

Further down the rabbit hole we go!

The frei0r build issue was caused by the opencv related libs on Squeeze being too old (libcv2.1, libcvaux2.1, libhighgui2.1) so I rebuilt the entire OpenCV stack to the latest 2.4.2 version and packaged it. Sooo... we have a working Kdenlive 0.9.2 (actually 0.9.3 from GIT) AND now we have the correct dependencies in place for people to be able to run the build scripts themselves (at least as the build script is currently) installed by default.

Looks like Kdenlive is back on the menu!!

Thanks to davephillips, trulan and also jade_1_4 for a little extra motivation...

[trulan]

I say forget Debian - let's just compile the whole OS from source!!     Oh wait, then we'd be running Arch or Gentoo - nah, that's no good...

Great news G, and great work as always!

[canezila]

Outstanding work!

[RickBrock]

KDEnlove  on the menu!

[jade_1_4]

Looks like Kdenlive is back on the menu!!
Thanks to davephillips, trulan and also jade_1_4 for a little extra motivation...

Thank YOU GMaq for so much passion and perseverance in producing the lightest on hardware, most user-friendly, full featured  art-distro ever ! Do bring out the 6.0 release with the new Kdenlive and if it runs as stable as we are used to, the additional $25 are yours instantly. BTW those interested in OSS video editing might want to take a look at this information-packed 6 part tutorial on Kdenlive, written by someone who works at a film production facility http://opensource.com/life/11/11/kdenlive-part-4-colour-correction

[kalimerox]

@ jade_1_4: this is a great kdenlive tutorial, thanks for the link!

(checking the forum 4 times a day for 6.0  )