Virtuous blogs jbarnes' braindump

10/26/09

English (US)   So I followed Paul Mundt into this narrow alley...  -  Categories: Announcements [A]  -  @ 11:15:47 am

Back from Japan at last (I think United lost my sleep schedule on the way home though, trying to retrieve it this weekend has been a challenge).

Both KS and JLS went well I thought. It was really good to connect with some of the Japanese developers that until now I’ve only interacted with through email.

The summit went well this year I thought. We didn’t have a big set of controversial issues to discuss, but we did sort out some development process issues. The highlight for me was the two customer panels. On the first day we had some people from TV and other vendors talk about how they’re using the kernel and other open source software. It’s interesting that some of them are stuck way back on 2.4 and very early 2.6 kernels. Part of the reason is long product development cycles, but mostly it’s because the SoCs used in many products only have support in a limited set of kernels (usually custom patches for specific kernels provided by companies like Montavista). The “platformization” work done by tglx and the x86 team recently (partly motivated by Intel “Moorestown” support, but also in preparation for more x86 based SoCs in the future) should help with this for x86 stuff. We definitely want to avoid an ARM-like situation where each SoC requires a specific kernel with incompatible firmware and hardware support. I had some good discussions with Linus and Paul on that topic; the tricky part will be ensuring that vendors adhere to some level of standardization in their platform and firmware support. Doing so will have big benefits: upstream kernel support should be better and much more flexible (good for the SoC vendors and their customers), and the platform maintainers should have a much easier job integrating support for new platforms without a huge set of ifdefs and incompatible firmware interfaces. Managed to get a few bugs fixed at KS as well, Ted & Dirk didn’t have anywhere to run when I wanted them to test some patches for problems they’d reported!

The JLS conference was interesting too, with a few good talks on things like barcode delivery of oops info and btrfs

Tokyo is a pretty amazing city. This was my first trip to Japan and a few of us were fortunate enough to have Paul Mundt guide us for a couple of evenings to explore the city. The narrow alleyways and tiny bars in the Shinjuku (at least I think that’s where we ended up) were really fun. We even checked out a Mexican bar called Bonita; Mexican stuff outside the southwest US and Mexico is always interesting, but the Japanese mix made things even more so. Overall a fun night including Japanese Denny’s food, passed out salarymen, and an everything store with some bizarre costumes, including some furry outfits we were tempted to buy… A bit later in the week we had a contrasting experience by going to Seamon (one of the dozens of one star Michelin sushi restaurants in Tokyo) and a high end scotch and cigar bar afterwards.

Ok now back to catching up on the huge backlog of patches that have accrued due to travel neglect.

09/21/09

English (US)   Off to Portland  -  Categories: Announcements [A]  -  @ 01:51:25 pm

Heading off to Portland tomorrow for Plumber’s (Wed-Fri) and XDC (following Mon & Tue).

I’m hoping we can get the compositing & GLX architectural improvements for X nailed down (see http://dri.freedesktop.org/wiki/CompositeSwap and our talk outline at http://linuxplumbersconf.org/ocw/proposals/70). My implementation of some of those features is already limping along and I’m hoping we can land it soon and start on some compositor improvements to take advantage of the new features.

The merge window has been busy so far, with a few good PCI improvements landed (including VGA arbitration, finally!), and a bunch of gfx related stuff: a slew of power management improvements (dynamic render, display and refresh rate controls, framebuffer compression, RAM self-refresh mode enabling and a bunch of clock gating enables), several stability improvements (GEM memory shrinker, bug fixes for our ring management, reloc range checking, automatic GPU reset support), and a few performance improvements (madvise support for GPU buffers). Overall very busy and very cool stuff. I’m still holding out hope we can land the page flipping and execbuf2 code this cycle; if the merge window stays open through Plumbers that should be possible.

08/28/09

English (US)   KDB+KMS FTW  -  Categories: Announcements [A]  -  @ 04:17:14 pm

alternate link: http://www.youtube.com/watch?v=aj6M2lR19iY

07/08/09

English (US)   Morning in America^WLinuxland  -  Categories: Announcements [A]  -  @ 12:57:16 pm

It’s been about two months since my last update. Most of last month was spent travelling, first to UDS in Barcelona (one of my favorite cities btw), then to London to work at OTC Europe on Moblin, and then on to Oregon to meet with the greater graphics team. In between flights and whenever I had time I also got some good work done; working through PCI patches, improving the error detect/collection code, testing the GPU reset patch a bit more, working on 3D tiling for pre-965 chipsets, and doing general bug fixing. But first things first.

UDS Barcelona

This was the second UDS in Spain I attended (first was in Sevilla a couple of years ago). Overall I found it to be a bit more organized and productive than the first. Probably a sign that the Ubuntu community and Canonical have grown a bit since then…

As I said, I thought this UDS was particularly productive, and ambitious to boot. Scott set an aggressive 10s boot time target for Karmic, and the desktop team decided to pull some very new technologies like KMS and DRI2 into the Karmic release. This is great news for users due to the improved feature set, but also makes life much better for upstream developers, since supporting the new code is much easier than dealing with old DRI1 stuff.

Another huge development (which actually pre-dates this UDS) is the xorg edgers repo. It’s a PPA containing packages of the graphics stack (kernel, libdrm, Mesa, X server and drivers) directly from git. Having this available means testing and development are greatly accelerated; now when users report a bug in the Karmic repos, we can ask them to quickly and easily test the xorg edgers bits to see if their issue has been fixed. If so, we know a backport may be needed, and if not we have a good bug report to feed upstream. I run this repo myself, typically updating every morning, and have found and fixed quite a few bugs as a result of finding them early. Robert and the rest of the edgers team deserve huge thanks from everyone in the Linux community for their work on this repo. I hope their example is followed by other projects, maybe for audio, bluetooth or wireless stacks, which also have large kernel and userland components.

Some other cool stuff got demoed at UDS this time, including some cool Android on Ubuntu work and an Ubuntu spin of Moblin! The latter is especially cool, and I’m hoping to install it on one of my Netbooks soon.

Graphics stuff

The London trip was something of a last minute affair. I was already in Barcelona, and our OTC Europe team in London was in the middle of working on some graphics related Moblin stuff, so they sent Eric, Ian and myself over there to help out. It turned out to be a very productive week; we fixed some major issues while there and overall improved performance by about 50% on some workloads. Some of that work will make its way into our next release and future kernels.

One of the big issues we worked on over there was implementing tiling for 3D textures. We did this awhile ago for some of the major buffers (front, back, depth), but doing it for textures is a bit more involved. Eric quickly got 965+ tiling working, but pre-965 turned out to be a bit more of a pain due to its fencing requirements. The 2D and display engines on pre-965 chips need fence registers to cover tiled regions in order to blit or scan them out. The 3D engine can handle tiled surfaces directly, without fences. The current execbuffer code (the central command submission mechanism) will always allocate fence registers for tiled surfaces however, and on chips with only 8 fence regs (915 and prior) that can be a problem. Not only are fence registers scarce, but mapping and unmapping objects with them can be an expensive operation. So I came up with the execbuffer2 interface, which adds a new relocation type to handle the fence register requirement. Commands using 2D blits for example can use a reloc type that indicates a fence register is required, while purely 3D commands can avoid it. There’s potential for more improvement if we remove 2D blits from the DRI driver, though that may involve more overhead than we’d like, due to the higher setup costs. As it stands, tiling textures can give us a ~20% performance boost on some workloads…

Another thing I had some time to work on at UDS and in London was GPU error handling. Our GPUs have some error handling capabilities we haven’t really taken advantage of until recently. Eric and Carl recently improved the GPU dumping utility significantly, which really makes GPU hang debugging possible. To make things even easier, I put together a patch to use the GPU error interrupt to trigger a error state capture and generate a uevent. The idea here is to capture the error state from the first error, then tell userspace it should capture a full ring & batch buffer dump as soon as possible. This should allow for automated reporting, ala kerneloops, of GPU related errors. The second part of the work involves automatically resetting the GPU when a hang is detected. I posted working patch for that aspect of error handling, but it still needs a little work on the hang detection side before we can push it upstream. I’m hoping both that and the execbuffer2 stuff will land in 2.6.32 (since it’s a bit late for big stuff to land in 2.6.31); the reset patch should be fairly easy to backport though.

Oh yeah, BUGS! The past couple of months have seen unprecedented bug fixing activity on the Intel graphics stack. The removal of the DRI1, XAA, EXA and much of the non-memory manager code is really started to pay off. We’ve been fixing bugs left and right lately, really stabilizing the drivers in both KMS and non-KMS configurations. Not having to worry as much about breaking some weird configuration possibility is a big help (though the non-DRM case still bites us from time to time). In short, things are really looking good on the graphics front these days; the major architectural changes are complete now, and we can really focus on making things solid.

PCI queue

And on the belated “what’s up with PCI this cycle” front, we had one major issue this time around. In an effort to keep the kernel from using BIOS reserved areas, we started using the ACPI _CRS (current resource settings) data from root bridges at boot time, to describe the set of resources on the root bus. Turns out some BIOSes list a ton of resources in _CRS, including legacy VGA and I/O port space. This can be helpful (since the alternative is having a huge list of chipsets and what ranges they’re hardcoded or programmed to decode), but the PCI layer isn’t quite ready to handle arbitrary numbers of bus resource ranges and types. So early in the cycle Linus had to revert the move to using _CRS by default; that said we did get some good fixes from Gary and Yinghai for the _CRS case, so eventually we should be able to use that data in some form.

Of course, eventually I’m hoping to add something like TJ’s resource management code to Linux. Unfortunately TJ seems to have disappeared, and he never did post his code. Fortunately he did leave a good set of notes on his website about what he’d discovered (e.g. which bits of info are reliable and how other OSes use ACPI data etc). The hard part is finding time to implement such a large change and get it tested well enough to feed upstream. The other perennial topic is VGA arbitration. Tiago recently updated his patchset for that, but I haven’t seen it formally submitted to the linux-pci mailing list yet…

05/07/09

English (US)   pageflipping, blocking, etc.  -  Categories: Announcements [A]  -  @ 08:23:56 pm

Lots of activity lately: Eric just landed a major cleanup to the 2D driver, there’s been lots of bug activity, and I’ve been hacking some more on the page flipping & DRI2 swap buffers support.

bugs

We still have way too many bugs open (but isn’t that always the case when the bug count is > 0?), but I spent a lot of time the last couple of weeks closing stuff out, fixing issues and generally troubleshooting configuration issues people have been having with the bleeding edge KMS, UXA and DRI2 code.

One particularly annoying issue was reported by Mateusz Kaduk. While we were debugging it recently, we found that for some reason IER (the interrupt enable reg of the GPU) was getting cleared sometime after the kernel driver was loaded (and had enabled it successfully!). During that time Dariush discovered that running vbetool to save the graphics state seemed to trigger the bug. So the VBIOS (which is ultimately what vbetool ends up running) disables interrupts behind our back, ouch! Take this as yet more evidence that the VBIOS really shouldn’t be run after you’ve booted and loaded a proper driver. Mateusz and I also worked out a workaround for the problem, which I posted here; not sure if we’ll actually ship that yet though, since the distro configs that called vbetool at boot time seem to have been fixed.

tearing

tear free shampoo

On the page flipping and swap buffers front, I’ve been having heavy discussions with Jakob Bornecrantz and Kristian Høgsberg lately about how things ought to work. I’m actually using one set of my code on the machine I’m typing on now, and it seems solid, but it has a few shortcomings we’d like to address. Overall there were a couple of issues we felt were important:

Beyond that there are the implementation details of how the new DRI2 protocol looks and what the exact sequence looks like on the display server and client side. Kristian is re-working things to require more of the display server (which should help Wayland), so hopefully we’ll see this work committed soon. It’s about time we had a way to enable tear-free compositing window managers.

Update: forgot to mention which bits I’m using:

other features

And since that wasn’t keeping me busy enough, I’ve been hacking on some other features lately, namely GPU reset and framebuffer compression for KMS. The first feature is tied in with some of the error handling improvements we’ve wanted for a long time. Recent GPU hangs (which are just plain hard to figure out) have motivated us to create some tools for dumping GPU command buffers, and to improve our handling of errors in the kernel. So I’ve got some code to capture error state when the GPU detects a failure, and also some code to reset the GPU which we can use if we encounter a hang or other fatal error. They both need a little more work though; we want to capture an error record right when we receive an error interrupt, so I need to create an error structure and export it through debugfs. Once that works reliably, I should be able to hook up the GPU reset code. That should make GPU hangs non-fatal; and if we’re lucky won’t even be noticeable to the end user. Framebuffer compression also needs a little more work; right now it just supports 965 and before; I need to add support for the G4x series and do some more testing to make sure it’s working as expected. Ah the fun never ends.

powered by b2evolution free blog software

Contact the admin - Credits: multiblog | green power hosting | test