Tuesday, December 22, 2009

Better display of wxTypes in MSVS debugger

This is a quick note about displaying a few wx types in MSVS (2003 and later) debugger as I just discovered, to some surprise, that some people using wx don't know about it. Let me show what I mean with an example: I have the following in my %VSINSTALLDIR%\Common7\Packages\Debugger\autoexp.dat:

... all the existing stuff ...
; wxWidgets types
wxRect=<x>,<y> <width>*<height>


This allows to see the contents of the variables of the types above in a much better way than the default display (e.g. by default tooltips for wxRects only show x, y and width values but the height is truncated -- hardly convenient). I wonder if others have more types to add to this list?

Of course, there are limits to this approach. For example, I'd really prefer to see an ISO 8601 representation of wxDateTime instead of the number of milliseconds since the Unix Epoch which is currently shown. But this probably requires writing a custom visualizer which is a bit more difficult (would anyone have done this already by chance?).

Still, debugging is much nicer when previewing wxStrings is as simple as hovering a mouse over them so if you don't have the above in your autoexp.dat you should really add it there. Just remember that wxString expansion is for 2.9, if you are using 2.8 you need to use <m_pchData,s> instead.

Thursday, December 10, 2009

Buzzwords Benchmarking

First, a word of warning: this post is not really about wxWidgets at all but I thought that my experience could be interesting for people doing cross-platform development and so might overlap with the interests of this blog readers.

The experience in question was the recent upgrade of my main Windows development machine. I need to develop for Windows but, at least in principle, it doesn't mean that I need to run it as my main OS -- personally I'm feeling more comfortable under traditional Unix-ish desktop environments rather than in Windows UI (as an aside, ever since OS X acquired virtual desktops support I hoped Microsoft would add it to Windows too, but apparently they don't copy the useful OS X features...). However traditionally I did use Windows (Windows 3.1 at the dawn of time, then NT 4.0 for more than 6 years and later Windows Server 2003 for about 5) just because using Microsoft Visual Studio is much more productive than cross-compiling from Linux, especially because of incomparably better debugging support. Nevertheless I always hoped to be able to switch to using Windows in a virtual machine under some Unix-like OS and this is what I tried to do with the latest hardware upgrade. The main question was, of course, how much slower would building software inside of a VM be compared to doing it natively -- so "virtualization" is the first of two buzzwords of the title that I wanted to benchmark.

The second one is "SSD". I've heard so much of how it's the greatest thing since the invention of computer from everybody starting from Anand Lal Shimpi to Linus Torvalds that I couldn't prevent myself from spending a good part of the new box price on a pitifully small (160GB) but hardly inexpensive Intel 2nd generation ("PostVille") X25-M SSD drive. And, of course, I was curious to know if this was money well spent for what matters to me -- that is, compiling/linking speed.

So I did some benchmarks on the new machine. They were perhaps not very thorough, e.g. I didn't reformat the drive after each run, however the variation remained small (1-2 seconds for the tests of physical systems) and the results were consistent so they still seem significant enough to me. The main caveat is that they clearly only make sense for this particular hardware configuration, it could be perfectly possible that using faster and/or more CPU(s) would change the balance. But I didn't have the possibility to test with anything else than this system, with its i7 860 CPU and 8GB of DDR-3 RAM (FWIW I also played a bit with RAM timings to see if this could change anything but they're absolutely insignificant in real-life benchmarks). The system also has 2 1TB Samsung F3 SATA drives which were used as baseline for SSD results (without RAID).

First of all, I compared the times for building wxWidgets under Linux. Here are some results under Ubuntu 9.10 (using g++ 4.4) and Debian Testing (using g++ 4.3):

WhatOSDisk/FSTime (sec)
make -j4UbuntuHD/ext491
make -j4UbuntuSSD/ext490
make -j8UbuntuHD/ext473
make -j8UbuntuSSD/ext470
make -j8DebianHD/ext366
make -j8DebianHD/ext263
make -j8DebianSSD/ext364
make -j8DebianHD/ext463
make -j8Debiantmpfs59
make -C samples/widgetsAnyany13

The first results seem to be obvious: SSD hardly presents any advantages in this test. I was rather surprised by this as I thought that compilation and especially linking was a disk-intensive activity. However it turns out that I was completely wrong and that in at least this hardware configuration it's almost entirely CPU-bound. This can be seen by noticing that even using tmpfs (which is clearly the most efficient disk you can use, provided there is enough of free RAM -- as was the case here) results in less than 10% gain. Also note that under Ubuntu there is no difference at all between HD and SSD when using 4 CPUs but a small difference in favour of SSD appears when 8 of them are used. This gives me some hope that maybe SSD can be useful with e.g. upcoming 6-core (and hence 12 logical CPUs taking hyper-threading into account) Intel processors. Or, even now, by using 2 or more Xeons instead of a single i7.

Next I installed Windows 7 (64 bit edition) and beta 2 of MSVS 2010. I won't give the benchmarking results for g++ under Windows as it's catastrophically slow compared to MSVC. Running configure alone took 100 seconds and building (with -j8) took more than 200. Instead I'll give some benchmarks for nmake (which doesn't support parallel builds) and msbuild (which can be used for building MSVS projects in 2010), both under the native Windows 7 installation and in the same installation (or at least produced using the same unattended setup file) in a virtual machine with 4GB of RAM and 4 CPUs under VMWare Workstation 7:

WhatOSDiskTime (sec)
nmake (x64)NativeHD162
msbuild /m:8NativeHD32
msbuild /m:4NativeHD34
msbuild /m:8NativeSSD31
msbuild /m:2VM7/LinuxSSD99
msbuild /m:4VM7/LinuxSSD57
msbuild /m:4VM7/Win7SSD67

There are two conclusions which can be made from the first half (under native installation of Windows 7). First, and the most important one for me, is that there is once again no difference between using HD and SSD to speak of. Second, amazingly, x64 builds are 20% slower than x86 ones. I don't know if it's due to x64 not being as optimized as the x86 one or just because x64 binaries are larger (by around the same 20%) and so take more time to generate.

The most interesting question however was if Linux could be used for developing under Windows efficiently. Unfortunately the numbers are not good. The best case is slowdown of almost 100% and I discarded the initial timings for the builds inside a VM: for some reason they take markedly longer time then the subsequent ones (10-15 seconds longer, both under Linux and Windows hosts). I don't know why exactly does this happen but it doesn't even really matter as even with the best results I still don't cherish the idea of spending twice as long waiting for the compile to finish. Of course, extra 20 seconds in case of wxWidgets build is not long at all but some of the other projects I'm working on take much longer to build and wait an extra half an hour is a big difference.

Notice that VMware Workstation only allows to use 4 CPUs (and not 8). I thought that the performance might be better if all CPUs were usable and so also tried 2 other virtualization solutions: KVM and VirtualBox. To kill any suspense, they were very far from working better than VMware. I didn't even benchmark KVM because I simply couldn't import my existing VM into it and spending time to reinstall MSVS 2010 in it just didn't seem to be worth the trouble because it felt sluggish enough interactively (probably because due to lack of any equivalent to VMware tools or VBox guest additions?) to not make me want to use it for longer than 5 minutes anyhow. KVM is also clearly very raw, I ran into (admittedly Debian-specific, but still) bugs while installing it and generally it just doesn't seem like something that is meant to work out of the box for interactive use right now.

VBox is another matter: it installed without any problems, imported my existing VM (after conversion to QCOW2 format) and allowed me to configure it with 8 CPUs. It also was perfectly usable interactively after I uninstalled VMware tools and installed the guest additions. All in all, it's a great way to use Windows under Linux... occasionally. Because the performance is just incredibly poor. I don't know if it's something SMP-related and I hope that this might improve in the next versions of VirtualBox (I used the last one available now, 3.1.0) but right now it's not even funny, just look at the numbers:

WhatOSDiskTime (sec)
msbuild /m:4VBox/LinuxHD750
msbuild /m:8VBox/LinuxHD1944

Using nmake is only about twice slower than under VMware (and 4 times slower than in physical machine) but there is clearly a problem with scheduling if building using 4 concurrent processes takes even longer than this and building using 8 of them is 4 times slower yet (that's half an hour build time instead of half a minute in a physical machine!).

To summarize, there are two conclusions. The most interesting one, if only because it was rather unexpected to me, is that if build times are your primary concern then you should spend your money on an extra Xeon instead of an SSD. I couldn't test it but it seems clear that building using 16 logical CPUs would be much faster than using 8 of them. SSD doesn't provide any advantage at all just because it isn't the bottleneck -- the CPU(s) is/are. Looking at perfmon charts under Windows it is quite clear: the disk utilization hovers around 10% with occasional spikes to 30% but CPU use is pegged at 100% during the entire build (when using msbuild /m:8). Of course, SSD have other advantages. As many people have noticed, booting is much quicker. And so is launching new applications. But personally I just don't care about either: I simply never reboot any of my machines and I mostly work with the same applications which are always running. I do notice that switching between different projects in MSVS IDE is much quicker when using SSD but IMO this doesn't justify the price premium. Undoubtedly I simply don't do enough of disk-intensive work but as much as I try I simply can't muster the same kind of enthusiasm as others have expressed after switching to using an SSD. The main advantage I see is the absolute quiet: although the hard disks are rather quiet as well, they do produce some noise when spinning up after a period of inactivity. But, again, this is hardly a game-changing advantage.

The second conclusion is less unexpected: compilation remains one of the probably worst activities for virtualization from performance point of view. This is somewhat surprising combined with my newly gained knowledge that it is CPU, rather than IO, intensive -- it would seem that it should virtualize very well because of this. But it doesn't, unless you count "twice slower" as being very well. Of course, this is not the end of the world and you could develop for Windows using a VM under Linux. It's certainly very usable and you almost don't notice any difference with the physical machine neither with VMware nor VirtualBox; I seriously considered just putting up with the extra wait (but finally couldn't use Linux on this machine anyhow because of the problems with resuming from suspend and so had no choice but to install Windows anyhow). OTOH you do need to buy VMware Workstation instead of using VirtualBox if you plan to do this (I probably need to add a disclaimer that I'm not affiliated with VMware in any way, I just love using their software even since version 3 and up to version 7 which I bought after doing these benchmarks, it's really invaluable for developing/testing and that you should, of course, compare their performance yourself just in case my case was somehow exceptional).

Sunday, November 01, 2009

Autumnal Tidings

I let slip the October post about the changes in the previous month so let me make this one, about combined changes in September and October, on time to try to compensate for it.

Many things have happened during these two months but the most important one was arguably the official 2.9.0 release. It's just the first release in 2.9 series and so many things will change (and already have changed since then) but it's still significant as it's our first non bug-fix release since 2.8.0 almost 3 years ago and the first one containing Unicode-related changes as described here, and, in much more details, in the updated Unicode overview in the manual (make sure to read about the changes since 2.8 if you're upgrading). Now you can easily download it and try it out if the idea of using an svn snapshot makes you nervous. Please do let us know about any problems you encounter when porting the existing code working -- it's not too late to do something to make it easier to upgrade to 3.0 but we need to know about the problems in order to fix them!

The rest of the changes were the usual mix of bug fixes (many) and feature improvements (a few). One of them stands out though: the long spoken, discussed, argued and even blogged about unification of the debug and release builds was implemented soon after 2.9.0 release. As with the Unicode changes, we're confident that it's for the better but it is also quite possible that these changes have introduced some problems in the existing projects/makefiles. And again, if you let us know about them, we'd do our best to fix them before the final 3.0 release.

As for the new features:
  • 2 of the 3 GSoC branches have been merged into the trunk so now wxRibbonBar and related classes as well as wxFileSystemWatcher are available in it.
  • New wxAny class by Jaakko Salli was added. This is a modern, more efficient and safer replacement for wxVariant and the name was chosen because of the similarity to boost::any. It is not used by wxWidgets itself yet but we hope to provide a bridge between it and wxVariant in the future and start using the new class in the API once all the problems with it are ironed out.
  • wxInfoBar GUI control was added. Instead of describing it, it is probably enough to show how it looks: there are two test bars in the dialogs sample now which can be shown or hidden using the menu commands but I'll spare you YouTube videos showing the effect and will just paste in three screenshots:

    (this is the native GTK+ version available with GTK+ 2.18 and later only, with earlier GTK+ it would looks similarly to the other platforms, notably it would show icons and position the buttons horizontally -- I have no idea why does the native implementation stack them vertically which looks rather ugly IMO) and

  • Many improvements in wxOSX/Cocoa port. Among cosmetic (but nice) things was the addition of ShowWithEffect() implementation, among less visible ones the addition of more conversions from various NSTypes to wx equivalents which makes developing wxOSX itself simpler. There were also improvements for iPhone (notably OpenGL-related ones) and fixes for 10.6. On a personal note, I started using Cocoa port instead of the Carbon for all new Mac development as I believe that it's in a good enough shape to be used now and prefer to fix any bugs in Cocoa code which will be used in the future and not in Carbon port which won't.
  • As usual, wxDataViewCtrl and related classes also received several bug fixed and improvements.

As usual, I won't describe the bug fixes in details but will just say that many of them were fixed both in the trunk and 2.8 branch so we'd really appreciate more testing of the current 2.8 branch in svn as the problem with fixing so many bugs in the stable branch is that this might inadvertently break something else and it would be great to find and fix it before 2.8.11 release rather than after it.

I will omit the statistics section this time as there doesn't seem to be much point, it doesn't change much from month to month, let me know if anybody was interested in it.

Well, that's all for now -- better brief than late.

Saturday, September 12, 2009

Debug Build Changes in wx3

This series of several recent commits finally implements important changes to debugging support in wxWidgets which we planned to do since a long time. To understand them better, it's probably helpful to say a few words about how the things worked until now, in 2.8 and all previous versions up to the recently released 2.9.0. So far we had two distinct builds of the library: a debug one and a release one. This should be familiar to Windows programmers as the dominant IDE under this platform uses the configurations with these names by default since many years but let me list the differences between the two for the benefit of others:
  1. Assertions are only enabled in debug build and disappear completely in release one: this is true both for the standard assert macro as it expands to nothing if NDEBUG is defined (which it is in release) and was also true for the wx macros such as wxASSERT which expanded to nothing unless __WXDEBUG__ was defined (and it was defined in debug builds only).
  2. Moreover, at least with the MSVC compiler, debug builds use the special debug version of C run-time library which not only has the asserts enabled in it but also has additional checks, notably very useful memory leaks and integrity checking which are not done in release builds.
  3. Debug information is only generated in the debug build on the (not quite correct as we'll see below) assumption that you don't need to debug the release build anyhow.
  4. In debug build, optimizations are turned off, including function inlining: this can have dramatic consequences for the performance of C++ code which uses a lot of trivial inline functions and adding a function call overhead to them means that the debug build is usually too slow to be used normally.
As you can see, this mostly makes sense for the programs. However it doesn't really for the libraries that these programs use as there are good reasons to use a release version of the library even in debug build of your application: e.g. you may want to to avoid getting slowed down by the lack of optimizations in the library code. And, conversely, you may want to have some debugging features even in the release build: assertions may still be useful (although they probably need to be handled differently) and debug information is invaluable when debugging crashes in production builds of your software (notably using the mini dumps produced by wxDebugReport). The debug/release separation is a heritage of the days when every CPU cycle and every byte on the disk counted and so production versions of the software couldn't allow to have any debugging checks in them but these days are long gone and being able to find a bug quickly is well worth the slightly increased library size and unmeasurable drop in performance.

Further, in the Unix world it's almost unheard of to have two versions of the library -- just check whether you really have two versions of any of the libraries used by wxWidgets itself, such as libpng, libjpeg or even different GTK+ libraries themselves. Nevertheless, at least the GTK+ libraries do provide some assert-like error checking even in the release build and you also can download a separate package with the debugging information for these libraries with many Linux distributions. In other words, they have the advantages of the debug builds without many of the problems.

And this is exactly what we have now for wxWidgets too. We still have two different builds under Windows because of the point (2) above: debug builds of the applications use debug CRT and so they must use a special build of the library which was also compiled with the debug CRT or unpleasantness (in the form of heap corruption when memory allocated by one CRT version is freed with another one) is sure to result. But this is the only difference between the two builds now. In other words, the "d" suffix in the library name now indicates only that it uses the debug CRT. Accordingly, the "d" suffix is not used at all under Unix (including OS X) as there is no debug CRT library there. This shouldn't be a problem from compatibility point of view as wx-config output has been adjusted to use the new libraries names so if you use wx-config in your makefiles nothing should need changing -- and if you don't, you really should consider using it. Under Windows the libraries names didn't change at all, of course, so there is no need to change anything neither.

On the other hand, the behaviour of the release version did change. All the details are explained in the updated debugging overview but, in brief, the main difference is that the asserts now remain in the release build too. They can be compiled out completely by using --disable-debug configure flag or setting wxDEBUG_LEVEL to 0 in wx/msw/setup.h explicitly under Windows but there should be no need to do this because by default the asserts are dormant in the release build of the application while they continue to produce the usual message boxes in case of an assertion failure in the debug build (again, of the application, not the library). In practice this means two important things:
  • Under Unix you can now use the release version of the library when developing. Unless you define NDEBUG when building your own code, the asserts will be generated just as they were before when you used the debug version so you will still profit from early error detection if you use the library in a wrong way. But you won't be hampered by slow operation of the library itself and you won't need to switch to another version for the release builds. Under Windows you still will need to do it, again because of the point (2), so nothing much changes there from this point of view.
  • But under both platforms you can now also enable asserts detection in the release builds. It is entirely up to you whether you're going to do it but personally I strongly recommend at least logging the assertion failures as they do indicate a problem in your code and may be valuable when trying to diagnose the problems your users experience. Showing the message box to your users is almost never the right thing to do though so you will want to define your own assertion handler using wxSetAssertHandler() function.
Inquiring readers may wish to know how is the magic with asserts being enabled for only debug builds of the application code is achieved. The answer is that wxWidgets fiendishly injects a bit of its own code into your application as part of IMPLEMENT_APP() macro. As it is compiled as part of your program, it can do different things depending on whether NDEBUG is defined or not in your makefile and it only disables the asserts if it is. Of course, it also means that if you don't use this macro in your code you do need to use wxDISABLE_DEBUG_SUPPORT explicitly as by default asserts are enabled.

There are two other, less important but still nice, things which have changed in the release build of wxWidgets: debug logging, i.e. output produced by wxLogDebug() and wxLogTrace() functions now remains in it too. Just as the asserts, it is disabled by default meaning that the cost of having the debug statements remaining in your code is extremely slight as their arguments are not even evaluated any more until you call wxLog::SetLogLevel() to enable it -- but you may do it if you need it.

And, finally, debug information is now generated for the release builds as well for MSVC. This is very useful if you use wxDebugReport as without it the dumps produced by it are mostly useless and doesn't have almost any drawbacks, in particular the executables size doesn't increase at all as the debugging information is generated in the separate PDB files so just about the only price you pay for this is the increased disk space consumption which shouldn't matter much in our days of multi-terabyte disks. We also plan to enable debug information for Unix builds in the future but we need to modify the makefiles to also produce it in separate files, just as MSVC does it, first.

To summarize, nothing much has changed for you, wxWidgets user, by default, and if you hadn't read this blog entry (which is excusable, especially considering its length) nor the list of important changes in docs/changes.txt (much less so!) you might not notice that anything has changed at all. However after reading this you should also know that you now have the additional flexibility of being able to enable asserts and/or debug logging even in your production builds and that you can use the same wxWidgets libraries both for development and production under Unix.

Wednesday, September 02, 2009

August News

An even shorter post than the one for the previous month as nothing much seems to have happened in August. Personally I mostly worked on other projects using wxWidgets and didn't have time to do much on wx itself and didn't even have time to finish my wxInfoBar contribution which was supposed to be done back in July. So it was mostly bug fixes and cleanup with some minor new features mostly contributed by others via patches on Trac.

August also marked the end of the GSoC. I'll try to write more about wxFSWatcher (or maybe invite Bartosz to write about it himself) later, when I integrate his work in svn trunk and would also like to invite other mentors to write about the projects they were involved in too but for now let me just say that all 3 projects were successful, so you can look forward to new wxRibbonBar and wxFSWatcher classes and AUI improvements in the next wx version.

Another good news is that people are working on using wx under several new platforms: this month we had posts about wxSymbian again (wxBase part only, no GUI yet), wxQNX (apparently wx already works under QNX in fact, although I don't even know which port does it use) and someone is writing a wxAmigaOS4 port. Personally I'd be most interested in the first one of those but more ports is always fun to have, even if AmigaOS is unlikely to ever become as popular as its predecessor was again.

The usual statistics for committers:

1. Vadim Zeitlin (106)
2. Jaakko Salli (11)
3. Stefan Csomor (10)
4. Mike Wetherell (5)
5. Paul Cornett (4)
6. Julian Smart (3)
7. Bryan Petty (1)
8. Jouk Jansen (1)
9. Kevin Ollivier (1)

show that the usual suspects were at it again: I was mostly checking in patches by others, Jaakko working on wxPropertyGrid and Stefan fixing bugs in wxOSX. Paul fixed a pretty bad bug in wxGTK which prevented it from working at all under Fluxbox WM (actually it was apparently a Fluxbox bug but users probably don't care about such fine distinctions).

101 new Trac tickets were created, 92 were closed but 10 were reopened so we seem to be in the red again -- but then it seems like it is the case every month so it's not very surprising any more.

Monday, August 03, 2009

July Summary

Another month, another post about progress in wxWidgets development. The most important recent change was covered by a separate post recently and I won't return to it but just say that wxLog is much nicer to use now in general and it's possible to use it effectively in multi-threaded programs, unlike before.

Other changes were mostly bug fixes (including a few important ones in wxSocket, but unfortunately it still seems broken under Mac, so I'll have something to announce the next month :-) and minor feature improvements. A huge patch globally replacing all occurrences of _T() macro with wxT()was applied so we finally got rid of one nastiness in wx code which is always nice (_T() is still available for user code for compatibility reasons though). And I also started working on wxInfoBar but it isn't quite done yet so this will be something for the next month again.

Except for this the most exciting new development wasn't actually in wxWidgets itself at all but outside of it when Ben Williams (AUI author) announced wxWebConnect availability. I didn't have time to try it myself yet but it definitely looks very promising and should allow to easily embed a modern HTML rendering engine (Gecko in this case) in wxWidgets applications. This opens quite a few possibilities, especially with the new features of HTML5 some of which are already implemented in Gecko.

As for the rest, as I can now use git as wxWidgets svn client, I also can use git shortlog which makes writing summaries simpler, e.g. here is the number of commits to the trunk per author:
  1. Vadim Zeitlin (129)
  2. Stefan Csomor (22)
  3. Jaakko Salli (14)
  4. Kevin Ollivier (4)
  5. Mike Wetherell (3)
  6. Vaclav Slavik (3)
  7. Jouk Jansen (2)
  8. Paul Cornett (1)
  9. Robert Roebling (1)
So you can see that in addition to my own changes which I described above, some work was done on OS X port and wxPropertyGrid. What can't be seen from here is that there were also several changes on 2.9.0 branch and a lot of ones in the topical GSoC branches.

Finally my script for analysing Trac statistics reports 109 new tickets and 67 closed ones of which 8 were reopened. So our goal of closing all the tickets didn't advance much this month neither -- help with them would be welcome!

Saturday, July 25, 2009

Playing with DVCS for wxWidgets

Just wanted to share my recent experiences with trying to use a DVCS for wx. I was interested in this because I often need to test some change on multiple platforms before committing it and currently what I do is to do the modification in a svn checkout on one machine, test it there, then make a patch, apply it to svn checkout on another one, test there, then commit from the first one, undo the patch on the other one and update from there -- this is not really complicated but certainly involves many more steps than I'd like. And this is the simple case when the patch actually works, if it doesn't and if you need to make changes to it, it's too easy to get entangled in multiple copies of the patch and get lost.

On the other hand, I'm using since some time Mercurial (also known as hg) for my own projects and enjoy its simplicity and how easy it is to create a separate clone of the repository for the changes. It's also so much faster than svn for a lot of common operations such as viewing the file history, annotating it or finding a given log message (which it can do by keyword, unlike svn). So I decided to try using hgsubversion, a bi-directional Mercurial-SVN gateway, for wx.

Unfortunately it didn't go great. Importing wx tree took a long time (~30 hours) and ran out of memory a few times as reported here, resulting in the process being killed by Linux out of memory killer (one of few times I've been actually glad to have it happen as a process consuming 8GB of memory without any good reason does deserve to be killed). Of course, this is a one-time only operation and so it doesn't matter much but, still, it was hardly a great start. More importantly, though, working with this setup turned out to be inconvenient in practice because cloning the entire wx tree does take time, even with hg efficiency, especially to a different machine. It can hardly be otherwise considering that the full cloned tree is 1.7GB -- it's not that much in absolute as svn checkout of the trunk (only) is 330MB, while hg tree contains all project versions and not just the latest one, but it's still a lot and, as we'll see below, it can be much better.

An alternative could have been to use Mercurial named branches but they are not meant for the private changes, i.e. using them would leave traces in svn history which is really not ideal (these branches are for my own personal testing and I do not want the others to see how many mistakes I made while doing a trivial change!). Or there is Mq extension which is supposed to be one of the greatest things about Mercurial but unfortunately I could never get used to it and it just doesn't seem right to me to use what is basically an orthogonal VCS on top of the one which is normally used. And the patch queue is local to each repository so with it I'd basically be reduced to copying patches around again. Maybe the most promising extension is the pbranch one, it really does seem to allow to do what I need. But it's non-standard, I'm unsure about its further prospects and it seems rather complicated thus negating the main advantage of hg -- its simplicity.

So, with a heavy heart, I turned to another popular DVCS: Git. I think it could be described as Mercurial evil cousin. While Mercurial is as easy to use as it could be and has great documentation, Git is almost perversely complicated. It has concepts which are particular to it only (can anyone really explain what purpose does the index existence serve except for confusing new users and occasionally tripping more experienced ones?). Its included documentation is only useful if you already know very well what you are doing. It allows (I think it encourages, really) you to make errors -- which is, of course, fine, as there are 3 or 4 different ways to undo them. Of which 2 (different ones, depending on situation) make things even worse. It seems to enjoy reusing commands commonly used in other VCS to do something different. Even the commands which seem to do what you'd expect (e.g. pull and push) do not. Moreover, they are not really even opposites of each other. So you never know what a command with a simple name does and you never risk finding any other commands without reading half a dozen of git tutorials. And even then you have to remember that the equivalent of hg histedit is git rebase -i (with rebase in general doing something completely different, of course). And using git means having one extra letter to type for every command compared to hg!

So ever since I found Mercurial I never seriously considered using Git. While I agree that Git is more powerful, having 37 different ways to shoot oneself in the foot is not really what I'm looking for in my VCS. Unfortunately, Git does have one killer feature: local branches. This is exactly what I need when working with wx svn and is close to what Mercurial pbranch extension does. Except, in this particular case only, Git is actually simpler. And faster.

Speaking about faster: importing wx svn using git-svn took "only" 12 hours. And never consumed any appreciable amount of RAM. And, a really pleasant surprise, the git repository of wx is only 400MB -- that is hardly bigger then svn checkout of a single trunk revision (while git repository, like the hg one, contains all versions of all branches in the project) and more than 4 times smaller than hg. In spite of myself, I was impressed. Think about it: this means that if you have both 2.8 and trunk checkout of wx you actually save 200MB of disk space by using Git -- while gaining all the advantages of having the entire project history locally (which is the reason for which switching between 2 branches in git is practical but using svn switch is not). And if, like me, you have 4 branches checked out (2.8, 2.9.0 (well, hopefully not for much longer, this one), SOC2009_FSWATCHER and trunk), the space savings becomes really noticeable (almost 1GB).

But it gets better: "cloning" (creating a new local branch) with git is instantaneous. Switching to another existing branch (e.g. 2.8 one) is much faster than with hg. Even updating from svn seems to be faster, although here the difference is not really significant (using the usual hg pull instead of git svn rebase is significant advantage of Mercurial though -- but unfortunately it's easier to get used for idiosyncratic syntax (yeah, and committing is done with git svn dcommit -- I'm sure there is a logical explanation for this extra "d", too...) than to slowness).

So I'm using git as my svn client for now (all of 2 days). And I'm ashamed to say I love it. Of course, hg is great compared to svn too. But I can't realistically use it with svn right now and I can do it with git. And so I don't have to jungle with patches any more. And the coloured output of git diff is so much easier to read than svn diff (and even than hg diff with colour on, as git also nicely highlights white space errors). Now if only I didn't forget to use that --cached option half of the times...

To summarize, I wholeheartedly recommend using Git as a client for wx svn repository. If there is any interest in it, I could push my repository to Github (it's bigger than their 300MB limit for free plan but I hope they could make an exception). But even if you need to run git-svn yourself, it's still great to have a local git repository if you plan on submitting (or even just having them privately) patches to wxWidgets. Of course, any DVCS could be used to have this extra freedom of working with wx in any way you want. But while I still hope hg implements local branches in the future and hgsubversion improves (there doesn't seem to be much point in hoping that git interface becomes logical), for now Git is the best choice of a DVCS to use with wxWidgets.

Monday, July 13, 2009

Blogging about logging

I've just finished a series of changes which were meant to make wxLog less embarrassing and more useful. Of course, wxLog was always meant to be a simple logging framework adapted for typical logging patterns of GUI applications but there is such thing as being too simple and it became apparent since quite some time that wxLog was insufficient for any kind of application using multiple threads or even simply separated in multiple components whose logging should be controlled simultaneously. And as most applications nowadays do use multiple threads, this is a serious limitation indeed.

As an aside, when I realized that the deficiencies of wxLog really prevented it from being useful in the application I was working on, my first idea was not to enhance it but to switch to another, dedicated logging library. But incredibly enough I couldn't find any good candidate: there are tons of libraries based on log4j but translating Java API in C++ is really not a good idea and I hoped to find something more idiomatically C++-ish. So I naturally turned towards Boost and found not one but two libraries named "Boost.Log", with one even confusingly called "Boost.Log v2" despite being older than the other one. Unfortunately, while both of them are undoubtedly great libraries, I was completely overwhelmed by their complexity. They are certainly great and allow some things I wouldn't even think of if I were creating a new logging library from scratch, e.g. a possibility to associate a decrementing counter starting from 100 with step of -5 with every log record which is extremely impressive but also doesn't seem to be especially useful in practice and I'd prefer to just simply use a logging library instead of admiring its marvellous elegance. So I passed them too -- and decided that while wxLog might be too simple, keeping it simple enough was still very important.

With this in mind, I decided to simply fix the few most glaring omissions in wxLog:
  1. Lack of support for logging from threads other than main.
  2. Impossibility to treat logs from different parts of application differently.
  3. Absence of __FILE__, __LINE__ and __FUNCTION__ information.
The first one was already solved for some logging targets, e.g. wxLogWindow was already thread-safe as it collected the messages coming from other threads and really displayed it in its text control only during the idle time from the main thread. All I did was to extend this approach to all log targets by moving its implementation in wxLog itself.

This does introduce a new problem however: as the messages are buffered instead of being output immediately, they could be lost if the program crashes before the main thread has a chance to output them. So I also added a concept of per-thread log targets which can be associated with a single thread only and don't need to do any buffering. Of course, such target can't show messages to the user -- as this can only be done from the main GUI thread -- but it can log them to a file and so a thread can always set up wxLogStderr or a wxLogStream to ensure that its messages are saved in a file as soon as they are output.

On a related note, using wxLogNull (and wxLog::EnableLogging() which it uses internally) now only disables logging for the current thread and not the application as a whole. This makes sense as if you just want to suppress an error message from a wxWidgets function you're going to call, you shouldn't disable all the logs from the other threads of your application which can be doing something completely unrelated while this function is executing. The initial plan was to also add a new way of disabling the logging globally but after thinking about it for quite some time I couldn't find any realistic use case when doing this would be really useful so for now logging can only be enabled thread-wise -- but we can always make it possible to disable it either globally or, which probably makes more sense, on log target basis, if really needed.

The second problem was solved by introducing the notion of "log components". These are simply arbitrary strings which identify the component which logged a message. By default, messages logged by wxWidgets come from the log component "wx" and its subcomponents, that is strings starting with "wx/" like, for example, "wx/net/ftp", while messages generated outside of wxWidgets have empty log component as it's not defined by default. This is already useful as sometimes you may want to treat wxWidgets and your own messages differently, e.g. you could disable all non-error messages from wxWidgets by setting the log level of the "wx" component to wxLOG_Error while keeping all messages, including the debugging ones, from your code enabled. But this feature becomes really useful mostly when you do define your own custom log components. This is done simply by #define-ing wxLOG_COMPONENT before using wxLogXXX() functions. It can be done on the compiler command line (to ensure that the same value is uniformly used everywhere) or inside the source files. In either case you will probably want to use different values for different parts of your application, e.g. "myapp/ui" and "myapp/db" and "myapp/network" and so on. And then you can independently configure the log level for each module and, also importantly, you can distinguish between the messages logged by different components and send them to different final destinations (e.g. database-related messages to one log file and network ones to another) from your overridden wxLog::DoLogRecord().

Finally, to solve the last problem in the list, all wxLogXXX() functions have been replaced by macros with the same names, which allows to record the information about the log message location. It can be retrieved from DoLogRecord() from the wxLogRecordInfo struct passed to it. By default, this information is not used in any of the predefined loggers (yet?) but it's available in case you nee it.

Moreover, in process of doing this, I actually created a relatively generic mechanism for passing arbitrary extra information to the log functions -- but, still remembering my experience of reading Boost.Log documentation, I decided to not make it public for now and to keep things simple.

After all, with the additions mentioned above wxLog is already much more useful and hopefully it's good enough for even complex wxWidgets applications now. And if not, we'd be interested to hear about still missing features, of course, so do have a look at the improved wxLog version in svn trunk and let us know what do you think!

Sunday, July 05, 2009

June News

Here is a brief summary of changes in wxWidgets during the past month. Once again, we (and I in particular) didn't do time to do as many things as we'd like to but less is better than nothing. And, in case of the most important new feature added, later is hopefully better than nothing as I seem to remember requests for it at least 10 years ago -- and now, finally, we do finally have ... drums roll, please ... support for images in wxButton:

(the images, and hence button sizes, are different in the screenshots above because the standard wxART_INFORMATION icon is used which is platform-dependent).

Another important even if somewhat technical change was the harmonization of handling of different background styles under all (major) platforms, as discussed here. As a side effect, this allows background bitmaps in wxHtmlWindow to work again in wxOSX/Carbon.

Other miscellaneous changes:
  • Some wxFont convenient methods such as Bold(), Larger(), Smaller() and non-const versions MakeBold(), MakeLarger(), MakeSmaller() were added: see "Similar fonts creation" section in wxFont documentation

  • There were several additions to XRC:

  • wxDirCtrl gained support for multiple selections (thanks to Steve Lamerton)

  • wxStandardPaths behaviour under Windows is now more flexible, see its new IgnoreAppSubDir() method.

  • wxVariant was improved to support wxLongLong.

Speaking of wxVariant, Jaakko is currently working on the new, better, safer and more efficient replacement for it called wxAny. Any feedback about it would be very appreciated as we'd really like to make a class we wouldn't be later ashamed of (which is unfortunately a feeling I often have about wxVariant).

Finally, the usual statistics: there were 418 commits to the repository, 95 tickets were created or reopened and 70 tickets were fixed (hmm, I wonder if the label "progress" is this still applicable?).

Wednesday, June 17, 2009

Hints for Editing XRC with Vim

If you are one of those strange people who prefer to write XRC files manually instead of using one of the many GUI editors, and also one of the enlightened people using Vim as their text editor you may be interested in this hint: although Vim is smart enough to detect that XRC files are XML without any extra prodding (as the presence of header at start of each XRC file is enough for XML file type detection to work), things can be made more comfortable with a little extra effort.

Before doing anything else you need to modify your .vimrc or _vimrc (under Windows) file to detect XRC files as a separate file type. For this simply add the following line to it:

au BufNewFile,BufReadPost *.xrc set ft=xrc

I also like to start with XML boilerplate already filled in when I create a new file so I additionally have

au BufNewFile *.xrc read ~/vim/template.xrc

where the file template.xrc contains

<?xml version="1.0"?>
<object class="" name="">

Now, I'd like to do spell checking in the XRC elements which contain user visible text. For this I create the file ~/vim/syntax/xrc.vim (this works under Windows too, just use whatever Vim considers to be your home directory instead of ~) with the following contents:

runtime syntax/xml.vim

syn region xmlString start="\(<title>\)\@<=[A-Z0-9]" end="\(</title>\)\@=" contains=xmlEntity,@Spell
syn region xmlString start="\(<text>\)\@<=[A-Z0-9]" end="\(</text>\)\@=" contains=xmlEntity,@Spell
syn region xmlString start="\(<label>\n\?\)\@<=[A-Z0-9]" end="\(</label>\)\@=" contains=xmlEntity,@Spell

and enjoy Vim help with correcting your mipsellings (how did you notice I wasn't writing this post in Vim?). Notice that the region definition is not very elegant but this was the best way I could find to make it work: using \zs unfortunately didn't work.

Next, I also defined a couple of helpful macros to insert the common constructions into XRC. This is done in ~/vim/ftplugin/xrc.vim (which will be sourced automatically by Vim thanks to our file type autocommand):

runtime! ftplugin/xml.vim

nmap <Leader>o o<object class=""><C-M><Esc>kf"a
nmap <Leader>v o<object class="wxBoxSizer"><C-M><Esc>O<Tab><orient>wxVERTICAL<Esc>o
nmap <Leader>h o<object class="wxBoxSizer"><C-M><Esc>O<Tab><orient>wxHORIZONTAL<Esc>o
nmap <Leader>i o<object class="sizeritem"><C-M><Esc>O<Tab><flag>wxALL<Esc>o<border>5

Notice that this supposes that you have the XML editing plugin installed, notably it relies on it to close all the tags. But surely you don't edit XML in Vim without it anyhow, right?

Much more could probably be done but I find that the above already makes editing XRC much more comfortable. And I definitely can do it much faster in Vim than using any GUI I tried so far.

Thursday, June 04, 2009

Spring News

Unfortunately I didn't have time to write the April summary until the second week of May and by then it seemed too late so I decided to postpone it and write a combined April-May one. As usual, postponing things doesn't make them easier however and now I find myself unable (due to all the usual reasons of lack of time, partly due to real life getting into the way) to write the May summary properly neither. But better something than nothing so let me just make a very short one.

Most of the activity in wx development community during this time was centred around 2.9.0 release. It still didn't happen but we're now at release candidate 4 and it looks good -- or at least good enough to not justify postponing 2.9.0 any more -- so the final release should be expected very soon. Of course, if you haven't tried 2.9.0-RC4 (or bz2 version if you prefer a much smaller download) yet you're strongly encouraged to do it and post your feedback to wx-users mailing list.

Speaking of the mailing lists, this was the most important administrative change in our recent history: all the lists (including wx-users and wx-dev) have migrated to Google Groups as the current (virtual) server didn't hold the charge any more resulting not only in disruptions to the mailing list traffic but also to frequent Trac outages as everything on that machine slowed down to a crawl due to Postfix bogging all the disk bandwidth. Some people disliked this change but from our point of view there was simply no other solution as switching to Google Groups was much easier than paying for a better server and administering it ourselves.

The only drawback of switching (so far) is that we still don't have any solution for the gateway between wx-users and comp.soft-sys.wxwindows USENET group. Unfortunately Google Groups don't provide this feature and so someone still needs to host this gateway -- if anybody reading this can volunteer either resources or at least information about how to do it, it would be great.

Other than that the usual cycle of adding new features and fixing old bugs continued. Among the features I can remember the following additions:

  • Status bar tool tips shown when the text is truncated.

  • Better integration with the standard streams (thanks to Jonathan Liu): now you can wrap any wxStream as a std::stream and/or std::streambuf (we still need a way to wrap any std::streambuf as a wxStream to achieve perfect interoperability).

  • XRC improvements: wxListCtrl columns and item can now be defined directly in the resource files which is very convenient, especially for the columns (thanks go to Kinaou Hervé). And wxImageList support which was added to allow this is also available for the other controls using it such as wxTreeCtrl and various wxBookCtrl-derived classes.

  • A new wxMouseEventsManager class was added to abstract mouse handling in controls with items -- it doesn't seem like much but actually handling the mouse events properly, including respecting users mouse sensitivity options, is not that trivial. It still remains to modify the existing code in generic wxListCtrl, wxTreeCtrl and wxDataViewCtrl implementations to use it.

  • An already existing but private until then wxTextWrapper was promoted to a public class status.

  • More wxGrid tweaks: it's now possible to selectively disable resizing of individual rows or columns (despite the incredible (and confusing) richness of wxGrid API you couldn't do it before: it was all or nothing only).

  • Laurent Humbertclaude has submitted a patch adding table border width support to wxHtmlWindow which allows to have visually nicer tables in wx applications.

Finally, the GSoC projects seem to be all getting into their stride nicely. I don't know much about the other two ones but I'm very satisfied with the progress of the one I'm mentoring. Moreover, it seems that Bartosz is going to add some important enhancements to wxEventLoop which are not part of file system events notification project strictly speaking but which would be very useful to any wx applications which need to monitor anything at all (to be more precise, any file descriptor, HANDLE or CFRunLoopSource depending on the platform) while running.

That's all for today -- and I'll try to find time to write more about June changes and also to finally finish my long promised and long overdue post about Bind() soon.

Monday, April 27, 2009

Accepted Projects for Google Summer of Code 2009

This is a follow up to the previous post about wx acceptance into GSoC 2009. We have had many great proposals this year -- it was the best year for us so far from this point of view IMO -- and the only regret is that only 3 of them could have been accepted, I really feel sorry for some students who have clearly put a good effort in making their application. Nevertheless I can't really be sad about it because the 3 that were accepted look pretty exciting to me. There are:
I'm mentoring the first one of those so I'm naturally quite interested in it but the wxAUI project is clearly very much needed too and in spite of my initial opposition to the idea of including ribbon bar into wx, it turned out to be the most popular proposal among both students and wx users so it definitely will be great to have too.

Finally, just as during the last year GSoC, even if wx itself has only received 3 slots, there is also at least one other wx-related project in a different organization: Perl has one slot for creating Perl bindings for wxWebKit. And Audacity is participating in GSoC again as well and while their projects don't seem to be especially wx-related it is still nice to see wx applications being part of GSoC. OTOH it's a pity that none 30 Python projects seem to be wxPython related, especially as wxPython itself is not part of the GSoC this year but well, like this we have something to hope for the next year :-)

In the meanwhile good luck to Bartosz, Malcolm and Peter and looking forward to their contributions to wx!

Saturday, April 04, 2009

Marching Forward

Here is a brief and late report of changes in wxWidgets in March. Somehow it looks that once again not that much happened, although it certainly seemed like a busy month. But most of the effort was spent on not very sexy activities such as bug fixing and release preparation -- and 2.9.0 was finally postponed to mid-April in spite of all that so I can't even boast about a new release this month. This was a bit of disappointment but OTOH we'd like 2.9.0 to be usable in production, the ominous zero at the end notwithstanding, and so we decided to release it slightly later but with less problems.

Personally I was also upset by not managing to finish my work on debug/release builds unification before 2.9.0. I did check in the changes introducing wxDEBUG_LEVEL which allows us to more flexibly configure the presence of assertions and other debugging helpers in your build, but the default behaviour didn't change yet, it will now have to wait for 2.9.1 as we didn't want to postpone 2.9.0 for that long neither.

A lot of time was also taken by preparation for GSoC 2009 and discussion of proposal ideas first and the proposals themselves later. This is definitely not time wasted though as we have more proposals this year than the last one (although about the same as the years before) and, most importantly, their average quality is much higher so I'm really looking forward to working with students this year.

Anyhow, in short here is what happened at wxWidgets code level:

  • Added wxTextEntry::SetHint() which can be used to show a hint string (e.g. "Search") in an empty text control or combobox. This uses native XP/Vista support if available or a generic fallback otherwise.

  • Made wxFTP logging more flexible with wxProtocolLog (thanks to troelsk). We also should extend this to wxHTTP in the future.

  • wxSP_WRAP support was added to wxSpinCtrlDouble and wxSP_ARROW_KEYS was fixed too; alignment flags are now honoured for both it and plain wxSpinCtrl (thanks Andrew Radke).

  • wxImageHandler can now have more than one associated extension making it finally possible to use both .jpg and .jpeg (Ivan Krestinin).

  • Events for wxComboBox popup drop-down and close-up were added (Igor Korot).

  • wxString::ToCLong(), ToCULong() and ToCDouble() were added.

  • Many fixes to wxDateTime parsing methods which were broken in variously entertaining ways after rewriting them to use iterators instead of string pointers some time ago. wxLocale::GetInfo() was extended to return various date and time formats in the process and as the result all our wxDateTime unit tests finally pass in all locales.

  • A bad and long standing bug 9638 introduced by Unicode-related changes was finally fixed. Several other bugs related to handling of embedded NULs in wxString were fixed as well. And all string unit tests pass too in all builds (wchar_t, UTF-8, ...) now.

  • XRC error reporting was significantly improved, now the line number is given in error messages.

  • wxDocView code was streamlined a bit more and made less surprising to an unsuspecting application programmer.

  • Several wxAUI bugs were fixed as Ben had some time to work on it recently.

  • Many OS X enhancements/fixes from Stefan and Kevin Ollivier.

  • And some wxDataViewCtrl improvements from Robert, as usual.

The previously mentioned Bind() function was finally finished and documented and I'll describe it in more details in a separate post soon.

The usual monthly stats: 732 commits, 137 new tickets and 101 closed ones (oops, no bragging about moving in the right direction this time).

And that's all for now, hopefully we'll have more meaty news next month with a flurry of commits which habitually accompany the end of release freeze period.

Thursday, March 19, 2009

Google Summer of Code 2009

Excellent news: wxWidgets has been accepted to participate in Google Summer of Code 2009 again! Congratulations to the other 149 accepted organizations but I'm especially glad about this one :-) I admit that I was rather worried about it as we already were lucky enough to be part of GSoC during the last 3 years and it was mentioned that some previously selected organizations couldn't be accepted this year to make place for new ones. But all is well that ends well.

Now we need to attract enough good students applications before the April 3 deadline to be able to choose the best ones of them. And while we also have a list of proposed projects we also are always looking for more so please update the Wiki page or post to our mailing lists if you think you have an interesting idea.

So if you are a wxWidgets user or a user of any wxWidgets application please tell any students around you about the possibility to work on wxWidgets during this summer. And, of course, if you are a student yourself, consider applying!

Monday, March 09, 2009

wx Sightings in the Wild

I've just discovered (please don't laugh as apparently everyone else made this discovery a year or at least half a year ago but better late than never) a nice solution for synchronizing files between different machines called Dropbox. For those of you who are as ignorant as me, it's basically a way to transparently rsync files (up to 2GB for free) with Amazon S3 storage. The transparent part is nice as it means that you just run a daemon monitoring the dropbox directory in an efficient way (using inotify under Linux and, my guess would be, using ReadDirectoryChangesW() under Windows) and don't have to manually commit and update your files as I was doing so far, using a central CVS server (from which I'm slowly moving to an hg one). And, unlike CVS, Dropbox also provides a nice way to access your files from any JavaScript-enabled browser (which is more convenient than hg web interface but then it's not at all meant to do the same thing).

The only problem with Dropbox is that the daemon part of their product is not open source. This is really a pity but it also explains why I started looking at its internals at all -- only to discover that it is written using wx, more precisely wxPython. It's not the most demanding GUI application in existence but it's still nice to see that an application used by many (how many? I don't know but judging from amount of stuff written about it on the web this must be one of the most widely used wx applications in existence) people is written in wx and apparently nobody complains about its GUI.

So while I'll probably mostly continue to use hg myself, I'll certainly recommend it to my less DVCS-inclined friends and relatives. And if any of the readers of this blog want to try it out, don't hesitate to use my affiliate link to register and get extra 250MB of space for both you and me.

Saturday, February 28, 2009

February 2009 News

Maybe it's because February is so short or, more likely, because we started a lot of things in January, but not that many exciting things have happened in wx land during this month. To continue from the last month summary, we did move closer to finalizing the new event handling code, although we had to introduce a new Bind() method (which I'm going to describe in another post soon) as trying to introduce new features while keeping backwards compatibility with the existing code using Connect() proved to be too hard. And the wxYield-related changes were mostly finished too.

Unfortunately not much has been done for the remaining big item for 2.9.0 release: unification of the debug and release builds. This is something which would be really useful for both the library users and developers and actually doing it will probably be not that difficult -- but deciding what exactly should be done proves to be more agonizing than expected. I still hope to do it in March but then I said the same thing about February at the end of January, of course...

There were also some discussions about releases: both 2.8.10 and 2.9.0. Please help us with testing them if you can!

The only new features of note implemented in this month were, AFAICS, the new wxTaskBarIcon implementation for wxGTK by Paul which should look much better in modern desktop environments and support for hints (a.k.a. cue banners or placeholder strings) to wxTextEntry (and hence wxTextCtrl and wxComboBox).

As usual, several bugs were fixed. The most important one was arguably #626 which is a problem with wxTreeCtrl selection events in wxTR_MULTIPLE mode. As can be seen from its number, it was a very long-standing ticket and while the fix hasn't been checked in just yet, it's going to be done very soon -- thanks a lot to Jonathan Liu who spent a lot of time working on this (and several other bugs as well).

Finally, some statistics for February: there were 632 commits in the repository, 67 new tickets were created, 84 were modified and 61 were closed. We'll try to do better in the next month (and preferably not by making Trac crash with internal server error and thus making it impossible for people to enter new tickets... I do love Trac UI and workflow but it's still amazing that it's the only ticketing system which I ever saw crashing -- but then I saw it do this all the time).

That's all for this month folks and have a nice March!

Saturday, January 31, 2009

January 2009 Progress Report

As the post about wx progress in 2008 seemed to generate quite some interest, here is the promised update for the things we did in the first month of 2009.

The main new addition was probably the long-discussed support for persistent controls. This is not completely done yet but initial implementation of the basic support for persistence was checked in and now persistent adapters for more wxWidgets classes (currently only wxTopLevelWindow and wxBookCtrls are covered) can be added one by one.

Another important change was the integration of the type-safe events patch from Peter Most. Not only it makes the events safer but it also allows connecting anything as an event handler, not necessarily a method of wxEvtHandler-derived class. Unfortunately we hit some problems with building this code with all but the very latest compilers (we didn't even try to use it with VC6, but it was a bad surprise that VC7 template deduction was not smart enough for it neither) so currently it is still disabled but we plan to re-enable it a.s.a.p.

Speaking of the improvements to compile-time safety, Francesco has modified wxWindow::ProcessEvent to be protected at wxWindow level as it should never be called directly but only as win->GetEventHandler()->ProcessEvent() or, with the latest trunk, as win->ProcessWindowEvent() which does the same thing, to avoid skipping any event handlers pushed onto the window. Doing this allowed to find several places in wxWidgets itself where GetEventHandler() call was forgotten and there are certainly a lot of similar omissions in code outside of wx itself which this change will now allow to find during compile-time.

There was also more work done on the native headers (with nice support of sorting and column reordering) in wxDataViewCtrl and wxGrid (special thanks to Zen Fire for sponsoring this work!) and on wxDataViewCtrl in general. In particular the simple wxDataViewListCtrl wrapper was added and event-based API for drag-and-drop was implemented.

The last problems with support for custom controls in native file dialogs under Windows were solved thanks to Marcin so now only Mac support for this feature is missing. Speaking of Mac, the new OS X port kept progressing in leaps and bounds and I'm especially glad that several people have contributed to it recently -- thanks!

Then there were some improvements to MDI stuff. This code was not updated for a long time so I started reviewing and correcting it in last November and continued to do so, while adding some new features, now.

And finally some useless statistics: so far (the month is not quite over yet...) there were 850 check ins in January. 138 new tickets were created, 211 ones were closed (at least we're moving in the right direction!) and Trac records 365 ticket modifications in total from 93 different people (with Robin and me doing most of the damage with about 70 changes each).

On a completely unrelated note, I've installed, out of curiosity, beta of Windows 7 and tested some wx samples under it. They have no special support of Windows 7 (nor even Vista yet) but they still appear to be quite fine, only small details (e.g. owner drawn menu items) look out of place. I'm not sure if this can really be seen as our merit though, it's probably more a compliment to Microsoft for maintaining backwards compatibility. Although it also clearly shows the advantage of using native controls -- I doubt that the programs written using Qt, especially using its version from a few years ago, look as good as the ones written even very old wx versions under the new system. Other than that, my main impression of Windows 7 is that it's amazing that it still needs to restart when changing the workgroup name. I guess they didn't want to remove all old and familiar features at once...

And to end this post, some less technical news: we made a brief roadmap for 3.0 release. This is not much in itself but it does help to concentrate thoughts on it and hopefully gives a bit more visibility into the project future for people not reading this blog.

And I was extremely glad to learn that wxWidgets now has a new maintainer in Debian as I almost lost hope to ever see them there as the old maintainer spent his time only sabotaging the efforts by others to bring wx to Debian and spreading FUD about it in his spare time whereas the new maintainer has already released the updated packages which is great news for me both from a wx developer and a Debian user viewpoint. We also had a discussion with Dan Horák, Fedora packager of wxGTK, about supporting multiarch (mostly x86/amd64) packages and making RPMs for wxWidgets 3.0 (interested people can already try these experimental pre-2.9 versions).

Wednesday, January 14, 2009

Qt switch to LGPL

Qt switch to LGPL has triggered a thread with subject "Bad thing for wxWidgets Qt will be in LGPL" on wx-users mailing list which clearly implies that this is indeed a bad thing. But is it one really?

First, let's not be egoistic: even if this were bad for wx, it's surely excellent news for just about everybody [else]. I do applaud Nokia for doing the right thing and I'm personally very glad that even people in big multinational companies such as Nokia understand the benefits of open source.

Second, this certainly will bring more users to Qt. But I'm not sure it's going to happen at the expense of wxWidgets users. I rather think more people thinking to use Java or C# or C (with GTK) could reconsider their choice and use C++ with Qt. The reason for this cautious optimism is that the cost of Qt has never been the only reason for choosing wxWidgets over it. Of course, this hasn't ever been important for GPL applications anyhow. And, surprisingly, it often wasn't very important for commercial applications neither as companies often choose to pay for wxWidgets support anyhow even if the library itself is free. So I think that relatively few of people who had chosen wxWidgets in the past would change their mind now, as the other two main reasons for this choice -- the use of native widgets by wxWidgets and the absence of any preprocessor -- remain as valid as ever.

Finally, if more people use C++ for developing cross-platform applications this is good for wxWidgets even if they use Qt because the whole idea of writing portable programs is unfortunately still not as entrenched in mass conscience as it should be in our opinion. So this could yet turn out to be positive for wxWidgets, although more likely it's not going to change much for us one way or the other.

But this remains great news for the open source community (of which we are part) so I just can't bring myself to regret it. I can dream about wxTNG which would combine the best features of wx and Qt but this still remains a dream for now...

Tuesday, January 06, 2009

Another Year of WX

I wanted to write this summary of the work done on wx in 2008 before the end of the year but, as it unfortunately often happens, ran out of time and couldn't do it. Still, better late than never -- again this is a very familiar principle -- and so here is a brief summary of important things which happened to wx in 2008. It liberally reuses the material from Robert's Wonderful World of WX write up, read it for more details.

So, what have we done in the almost 7000 revisions checked in during 2008? Maybe surprisingly, the most important changes haven't been about writing code at all but rather about improving the project infrastructure. This may not seem like a big deal but the old SourceForge-based bug tracker was completely unusable and basically was unused because of this and literally hundreds of bugs could have been triaged and closed since the switch to using Trac. We also finally have a working BuildBot which allows to detect breaking the build (and assigning the blame) quickly and automatically and this is very useful for a project such as wx where it's not always possible to test the compilation of the new code in all configurations under all platforms. Finally, the documentation now is in a new Doxygen-based format which makes it easier to maintain and contribute to it, as well as producing nicer results.

By the way, you might have noticed that one of the goals of these changes was to make it easier to contribute to wxWidgets:

  • Trac not only allows us to process bugs more efficiently but will hopefully also encourage others to to help with weeding out the duplicates (something we really couldn't expect the others to do with the SourceForge tracker as we couldn't even do it ourselves). And it allows for better bug categorizations, for instance you can see all easy to fix bugs at a glance and hopefully this can encourage more people to try fixing them (the list is short right now but will probably become longer with time).

  • The BuildBot allows to verify that the svn trunk version at least builds which is supposed to encourage people trying it out. We also hope to get more volunteers setting up their own build slaves to help us test even more configurations.

  • And the new documentation format has already allowed many more people to submit contributions to the manual than before and we hope that this trend is going to continue. And using Doxygen further lowers the barrier to entry to wxWidgets development as no LaTeX knowledge is needed any more.

But while waiting for the success of our diabolic plans to make wxWidgets users work on improving the library themselves we still found time to do quite a few things ourselves. From the strategical perspective the most important change of 2008 was certainly the decision to start merging the old Carbon-based wxMac port and the alpha-quality wxCocoa into the new, one and only, wxOSX port which will run on both 32 and 64 bit platforms and also iPods and iPhones and not just the boring desktops. And as a side note, wxOSX can now also be built as a framework and not just as plain old Unix-style dynamic library.

Another strategic decision was to concentrate more on the quality of the library than on the quantity of the things it may do. This has led to dropping of ODBC library which was embarrassingly buggy and unmaintained since a very long time and spending a lot of effort on bug fixing, including the inconsistencies in behaviour between different ports (and while this post intentionally doesn't cite any names to avoid accidentally omitting somebody, I do have to thank once again Google for organizing, once again, the Summer of Code which allowed us to spend more resources than would have otherwise been possible on this task -- let's give credit where it is due). In the same category, we have finally cleaned up wxSocket and wxURL code which became an acute embarrassment to all of us after a decade of bit-rotting and also fixed some, although by far not all, problems with wxGrid which also suffered from chronic lack of attention.

Of course, all this doesn't mean that there were no new features added so let me list the most important ones:

  • The first among them was the long-awaited inclusion of wxPropertyGrid in wxWidgets itself.

  • wxDataViewCtrl and related classes continued to be significantly enhanced.

  • wxRichTextCtrl has benefited from many small but cumulatively important improvements as well.

  • As part of improving wxGrid/wxListCtrl columns display support, a new wxHeaderCtrl class was added and a handy wxRearrangeList and related classes were added and column reordering was implemented.

  • A convenient new sizer class, wxWrapSizer, was added.

  • wxCalendarCtrl now has a native implementation.

  • The humble wxMessageDialog got a much needed face lift as well, in particular you can finally use custom labels for its buttons

  • We even found something to improve in wxStaticText which now supports "ellipsization" of its contents.

  • wxGLCanvas gained anti-aliasing (multi-sampling) support.

  • The main -- but far from only -- change to AUI library was the addition of wxAuiToolBar.

  • Events (and hence timers and asynchronous sockets) support was finally added to wxBase, after years of planning to do it.

And I probably forget quite a few more.

But while things have progressed quite a lot, we didn't manage to do everything we'd have liked to, of course (otherwise what would be doing this year?). My main regret is that we didn't manage to release 2.9.0 development branch preview release as I'd really like to do this a.s.a.p. to get more feedback about the Unicode-related changes. Among the features we hoped to add this year but didn't one really stands out: a lot of people are asking for images support in wxButton and we really should support this as it's not even difficult to do so this will definitely be something to do in 2009.

The two other small things we hope to see in this new year are the wxOSX completion and hopefully the 3.0 release. Let's see if this happens...

That's all for 2008 and see you in a year time (unless people find this post really interesting, in which case I might try to write monthly summaries of wx development)!

P.S. Edited to correct the date on 2009-01-09.

Sunday, January 04, 2009

Pretty printing wxStuff in gdb

I've recently stumbled upon a highly informative series of articles by Tom Tromey, one of gdb developers (among other things), about embedded Python in new gdb. The one immediately useful consequence of this is that you can now write pretty printers for custom data structures as explained here. Unlike the custom gdb commands, the custom pretty-printers apply to all occurrences of the objects, including as fields in the other data structures and even in the backtraces printed by gdb itself so they are much more useful.

So without further ado here is a pretty printer for wxString from svn trunk (it can also be trivially adapted for 2.8 version by using m_impl.m_pchData) shamelessly stolen from the examples above:

class wxStringPrinter:
def __init__(self, val):
self.val = val

def to_string(self):
return '"' + self.val['m_impl']['_M_dataplus']['_M_p'].string() + '"'

gdb.pretty_printers['^wxString$'] = wxStringPrinter

This is supposed to be put into a file sourced with source -p gdb command (there is apparently a better way to load this than simply inserting the command in .gdbinit but for now this is good enough for me).

And then:

[sunset:../src/build/wx-gtkud/samples/minimal]% gdb minimal
(gdb) b 149
Breakpoint 1 at 0x40cc65: file /usr/local/src/wx/HEAD/samples/minimal/minimal.cpp, line 149. (2 locations)
(gdb) r
Starting program: /usr/local/src/build/wx-gtkud/samples/minimal/minimal
[Thread debugging using libthread_db enabled]

Breakpoint 1, MyFrame (this=0x5751a0,title="Minimal wxWidgets App") at /usr/local/src/wx/HEAD/samples/minimal/minimal.cpp:149
149 SetIcon(wxICON(sample));
(gdb) bt 1
#0 MyFrame (this=0x5751a0, title="Minimal wxWidgets App") at /usr/local/src/wx/HEAD/samples/minimal/minimal.cpp:149
(gdb) p title
$1 = "Minimal wxWidgets App"

Of course, you do need a Python-enabled gdb for this to work. The easiest way is probably to get the package from Debian Experimental as I did. Otherwise you can always follow the instructions on Tom's blog and get gdb sources from git and built them yourself.

But this is really incredibly useful and will be even more so when we add pretty-printers for the other wxWidgets types (e.g. wxDateTime could definitely benefit). Any contributions to our custom pretty-printers would be welcome!