jbarnes' braindump


In memory of Eugeni

Filed under: Announcements [A] — jbarnes @ 10:03:32 pm

I’ve been out on sabbatical for just over a week now, and only just found out my team lost one of its number recently, Eugeni Dodonov.

Eugeni was a core contributor to our team, taking on new tasks and interacting with other groups and customers alike. He was both fearless and personable, always willing to help even while buried in difficult tasks (like the recent Haswell enabling). He was planning to be married later this year, and I was hoping to be able to attend his wedding, so my thoughts are with his fiance especially.

You’ll be missed, Eugeni, and remembered by all those you helped, your many contributions to various projects, and your especially positive and constructive attitude!


Writing standalone programs with EGL and KMS (updated)

Filed under: Announcements [A] — jbarnes @ 04:05:46 pm

Dear lazyweb,

Both on dri-devel and at the most recent Kernel Summit, the idea of a KMS based console program came up yet again. So in the interest of facilitating the lazyweb to write one, I thought I’d provide a review of what it takes to write a simple KMS program that ties in with GL, just in case anyone wants to port the VTE widget and give me my VTs on a cube. :) The ideal situation would be one where a distro could set CONFIG_VT=n and have userspace provide all console services, including support for the various types of input and output that exist in the wild today. The only thing that ought to be left to the kernel is spitting out panic information somehow. That means having a very simple text drawing layer remain, and a way for the kernel to flip to the latest kernel console buffer (today these functions are provided by fbcon and the DRM KMS layer, but if a userspace console service existed, we could likely clean them up quite a bit, and remove the need for the full VT and fb layers to be compiled in).


One of the first things any DRM based program must do is open the DRI device:

int fd;
fd = open("/dev/dri/card0", O_RDWR);

You’ll need to save that fd for use with any later DRM call, including the ones we’ll need to get display resources. The path to the device may vary by distribution or configuration (e.g. with a multi-card setup), so take care to report sensible error strings if things go wrong.

The next step is to initialize the GBM library. GBM (for Generic Buffer Management) is a simple library to abstract buffers that will be used with Mesa. We’ll use the handle it creates to initialize EGL and to create render target buffers.

struct gbm_device *gbm;
gbm = gbm_create_device(fd);

Then we get EGL into shape:

EGLDisplay dpy;
dpy = eglGetDisplay(gbm);

We’ll need to hang on to both the GBM device handle and the EGL display handle as well, since most of the calls we make will need them. What follows is more EGL init.

EGLint major, minor;
const char *ver, *extensions;
eglInitialize(dpy, &major, &minor);
ver = eglQueryString(dpy, EGL_VERSION);
extensions = eglQueryString(dpy, EGL_EXTENSIONS);

Now that we have the extensions string, we need to check and make sure the EGL_KHR_surfaceless_opengl extension is available, since it’s what we’ll use to avoid the need for a full windowing system, which would normally provide us with pixmaps and windows to turn into EGL surfaces.

if (!strstr(extensions, "EGL_KHR_surfaceless_opengl")) {
    fprintf(stderr, "no surfaceless support, cannot initialize\n");

And now we’re finally ready to see what outputs are available for us to use. There are several good references for KMS output handling: the DDX drivers for radeon, nouveau, and intel, and testdisplay.c in the intel-gpu-tools package. (The below is taken from eglkms.c in the mesa-demos package.)

    drmModeRes *resources;
    drmModeConnector *connector;
    drmModeEncoder *encoder;
    int i;
    /* Find the first available connector with modes */
    resources = drmModeGetResources(fd);
    if (!resources) {
        fprintf(stderr, "drmModeGetResources failed\n");
    for (i = 0; i < resources->count_connectors; i++) {
        connector = drmModeGetConnector(fd, resources->connectors[i]);
        if (connector == NULL)
        if (connector->connection == DRM_MODE_CONNECTED &&
            connector->count_modes > 0)
    if (i == resources->count_connectors) {
        fprintf(stderr, "No currently active connector found.\n");
    for (i = 0; i < resources->count_encoders; i++) {
        encoder = drmModeGetEncoder(fd, resources->encoders[i]);
        if (encoder == NULL)
       if (encoder->encoder_id == connector->encoder_id)

At this point we have connector, encoder, and mode structures we’ll use to put something on the display later.

Drawing & displaying

We’ve initialized EGL, but we need to bind an API and create a context. In this case, we’ll use OpenGL.

    EGLContext ctx;

    ctx = eglCreateContext(dpy, NULL, EGL_NO_CONTEXT, NULL);

    eglMakeCurrent(dpy, EGL_NO_SURFACE, EGL_NO_SURFACE, ctx);

Now we have a fully functional KMS and EGL stack and it’s time for a little bit of glue magic. We need to create some buffers using GBM and bind them into EGL images that we can use as renderbuffers.

    EGLImageKHR image;
    struct gbm_bo *bo;
    GLuint fb, color_rb, depth_rb;
    uint32_t handle, stride;

    /* Create a full screen buffer for use as a render target */
    bo = gbm_bo_create(gbm, mode.hdisplay, mode.vdisplay, GBM_BO_FORMAT_XRGB888, GBM_BO_USE_SCANOUT | GBM_BO_USE_RENDERING);

    glGenFramebuffers(1, &fb);
    glBindFramebuffer(GL_FRAMEBUFFER_EXT, fb);

    handle = gbm_bo_get_handle(bo).u32;
    stride = gbm_bo_get_pitch(bo);

    image = eglCreateImageKHR(dpy, NULL, EGL_NATIVE_PIXMAP_KHR, bo, NULL);

    /* Set up render buffer... */
    glGenRenderbuffers(1, &color_rb);
    glBindRenderbuffer(GL_RENDERBUFFER_EXT, color_rb);
    glEGLImageTargetRenderbufferStorageOES(GL_RENDERBUFFER, image);

    /* and depth buffer */
    glGenRenderbuffers(1, &depth_rb);
    glBindRenderbuffer(GL_RENDERBUFFER_EXT, depth_rb);
    glRenderbufferStorage(GL_RENDERBUFFER_EXT, GL_DEPTH_COMPONENT, mode.hdisplay, mode.vdisplay);

    if (glCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT)) != GL_FRAMEBUFFER_COMPLETE) {
        fprintf(stderr, "something bad happened\n");

At this point, you’re all ready to start rendering using standard GL calls. Note that since we don’t have a window system or a window bound, we won’t be using eglSwapBuffers() to display the final renderbuffer. But we still need to make sure the rendering is finished before we display it, so we can use glFinish() or some out of band synchronization mechanism (TBD) to ensure our rendering is complete before we display the new buffer.

Assuming we have something presentable, we can now try to display it:

    uint32_t crtc;

    /* Create a KMS framebuffer handle to set a mode with */
    drmModeAddFB(fd, mode.hdisplay, mode.vdisplay, 24, 32, stride, handle, &fb_id);

    drmModeSetCrtc(fd, encoder->crtc_id, fb_id, 0, 0, connector->connector_id, 1, mode);

In this example, we’re assuming a 24 bit depth buffer with 32 bits per pixel. That can of course vary, but what’s supported will depend on the chipset (24/32 is fairly common however). The drmModeSetCrtc call will also take care to simply flip to the fb in question if the mode timings are already programmed and are the same as the mode timings passed into the call. This allows for a simple page flipping scheme that simply creates two fbs using AddFB (pointing at different buffers) and flips between them with SetCrtc calls in a loop. Alternately, one can simply draw to the buffer that’s being displayed, if the visibility of partial rendering isn’t a problem.

DRM master privs and authentication

I forgot to mention that any program like this must drop DRM master privileges if another app, like X, wants to start up and use the DRM layer. The calls for this are drmSetMaster(int fd) and drmDropMaster(int fd). If no other DRM master is present, the SetMaster call should succeed. And before switching away due to a dbus call or whatever, any console program using the DRM layer should call drmDropMaster to allow the next client to get access to the DRM. Alternately, the console program could provide a display management API and make X a slave of that interface. That would also mean providing DRM authentication for other apps wanting to do direct rendering (typically master privs are required for display management, and simple auth privs are required to do command submission). Wayland is a good example of a display manager that provides this kind of functionality.

And that’s all there is to it, and should be enough to get someone started on kmscon… :) If you have questions or comments, feel free to bring them up at dri-devel@lists.freedesktop.org, in the comments section here, or via email to me.


Intel display controllers

Filed under: Announcements [A] — jbarnes @ 05:18:16 pm


Intel has been making graphics chips for a long time now, and the display controller has been updated in nearly every generation. Early chips supported VGA, then DVO was added to allow boards to include DACs to drive other output types (e.g. LVDS or TV). Over time, more functionality was pulled into the display controller, including LVDS, TV, and DVI support, along with an improved add-on interface called SDVO. The introduction of Arrandale and Core HD graphics is arguably the most significant change we’ve seen though, since rather than being a relatively straightforward integration of an existing interface, it splits display controller functionality between the CPU and PCH. In the past, both the GPU and display controller were part of the GMCH (graphics and memory controller hub), but with the memory controller moving onto the CPU, something had to give. So starting with Core HD graphics (code named Arrandale), most display functionality moved to the PCH (platform controller hub), with the display planes and GPU remaining on the CPU (in the same package at first, but on-die as of second generation Core processors).

The above roughly illustrates the split. As you can see, one output controller for eDP (embedded DisplayPort) is contained in the CPU, while the rest of the display functionality has migrated to the PCH. Between the CPU and PCH is FDI (flexible display interface), used by the CPU when something other than the builtin eDP output needs to be driven. Having eDP integrated onto the CPU provides both programming and power advantages on platforms that choose to use it (i.e. most eDP laptops without switchable graphics). Fewer clocks and links have to be configured (meaning less can go wrong on the programming side), and both FDI and the PCH display logic can be kept powered down, significantly reducing power consumption.


The CPU contains all the display data related functionality: display planes & pipes, framebuffer compression, panel fitting, cursor planes & control, video overlays, palette handling, color control, watermarks & FIFOs, and display interrupts. As mentioned last week, the display planes feed data to the display pipes, which in turn provide data to either the integrated eDP link or to FDI for passing to the PCH.

Pipes & Planes

On current Intel hardware, there are two display planes, each of which can drive pixels to its corresponding pipe. This allows two independent displays to be driven simultaneously, as in an extended desktop configuration for example. However, given clock and output limitations, sometimes both planes & pipes are needed for cloned configurations as well (e.g. eDP plus anything on the PCH).

Before a pipe/plane combination can be active, however, a clock source must be enabled. The clock source is used to drive the data transfer and other activity in the pipe/plane configuration. The PCH provides this to the CPU, providing a reference clock for both the on-die eDP controller and any FDI links that may be activated. Spread spectrum clocking is available as well, in configurations where noise and interference may be a problem.

FBC and power

The plane can optionally be compressed by the framebuffer compression (FBC) unit. If additional memory is provided to the FBC unit, it will periodically read the pixels in the display plane, compress them, and write them to a compressed framebuffer. If enabled, pixels will be fetched from this compressed buffer when the pipe needs data, rather than from the main display plane. This reduces memory traffic, saving power. Keeping memory idle is especially important when the memory can enter self-refresh (possible when the CPU is idle and no devices are performing DMA), since memory in self-refresh mode consumes far less power than it would otherwise. The display plane FIFO watermarks affect power consumption in a similar way; if set too conservatively, the frequency of memory traffic increases, preventing long periods of self-refresh mode. However if set too aggressively, FIFO underruns can occur, leading to display corruption.



Assuming a given configuration needs to drive something other than the on-die eDP link, FDI must be configured and enabled to provide data to the PCH. FDI is similar to DisplayPort, and despite being a fixed frequency, on-board link, it requires link negotiation and training, including vswing and pre-emphasis configuration. Fortunately, this is generally a very quick operation, so doesn’t contribute to noticeable delays in the mode setting sequence (especially not compared to some of the other delays involved, like panel power up). First the receiver and transmitter clocks are enabled, then the link is trained and enabled, allowing pixels to flow from the CPU to the pipe (called the transcoder) on the PCH, which in turn drives the configured display output. Although FDI is fixed frequency, minimizing the number of enabled lanes will prevent unnecessary power consumption.

Output ports

The PCH can drive many types of outputs directly: DisplayPort, LVDS, HDMI, and VGA. In addition, it can drive an optional SDVO link; this allows for other types of outputs to be connected, for example a TV encoder. The PCH also has logic for controlling an attached panel (whether eDP or LVDS), providing interfaces for both backlight and power control. Audio interfaces are provided as well, to support HDMI and DP audio functionality. The PCH contains some interrupt handling logic as well, which feeds to the master CPU interrupt status registers. This allows the PCH to notify the driver of hotplug events, FDI related features, DP AUX events, and transcoder related errors (e.g. FIFO underruns).


Debugging display problems

Filed under: Announcements [A] — jbarnes @ 01:34:28 pm

I’ve had the pleasure to debug several display related problems recently, and in the process found some time to put together something I’ve been wanting to do for a long time: put together a set of pictures and videos illustrating common display bugs.


But first, some background. What shows up on your display starts out as an array of pixels in memory. The exact format of these pixels varies, for example they may be 8 or 24 bits wide, or they may represent RGB or YUV values. These arrays of pixels are usually called “planes", and there may be several of them. The planes provide a source of data for a CRTC, sometimes called a “pipe". The pipe takes one or more planes of data (along with information on how to blend them, e.g. stacking order and alpha value for transparency) and sends them, with specific timings, to encoders, which are responsible for converting the stream of bits into something consumable by your display, which could be attached to a VGA or DisplayPort port for example.

Source formats

As mentioned above, the plane can be in one of several formats. There are several variables that apply: bits per pixel, indexed or not, tiling format, and color format (in the Intel case, RGB or YUV), and stride or pitch. Bits per pixel is as simple as it sounds, it simply defines how large each pixel is in bits. Indexed planes, rather than encoding the color directly in the bits for the pixel, use the value as an index into a palette table which contains a value for the color to be displayed. The tiling mode indicates the surface organization of the plane. Tiled surfaces allow for much more efficient rendering, and allowing planes to use them directly can save copies from tiled rendering targets to an un-tiled display plane. Finally, the color format defines what values the pixels represent. Planes dedicated to displaying video are often blended on top of the normal display plane, and sometimes use a YUV color format for convenience and compatibility with digital video formats. The stride (also called pitch) describes the width of each line in the source plane, either in bytes or pixels depending on the context.


Once the plane has been configured with the proper source format, and the data is ready to be displayed, the pipe must be configured to pull bits from the plane and send them out to the appropriate encoder(s). As mentioned above, pipes (in their function as “pixel pumps") need to be driven by a reference clock source with specific timings. These timings are derived from the mode to be displayed (resolution, bit depth, pixel clock) and from the requirements of the encoder to be fed. Based on these parameters, the driver will calculate PLL (phase-locked loop) values which will cause the pipe to run at a specific frequency. In addition to the pipe clock source frequency, the mode timings must also be programmed. These are split into horizontal and vertical components and used to drive each scanline sent out by the pipe. For instance the horizontal total (HTOTAL) value specifies the number of pixel timing widths present in each scanline (which includes time for each chunk of pixel data, along with time for various display related delays, like the time between the end of pixel data and the start of the next line of pixel data). Similarly, the vertical data contains information about how many lines are in a display, along with time between frames (the so-called vertical blanking period, during which no visible display lines are being modified on the screen). Pixel data is stored in a FIFO on its way to the encoder(s); this can help save power (as RAM is not always active during scanout) and buffer against high latencies that may occur if memory is busy or portions have been put into a low power state.


Finally the data arrives at encoder(s) (a pipe may drive one, or more, in the case of cloned display configurations). Encoders convert the pixel stream into a signal compatible with the display attached (e.g. from internal pixel timing signals into LVDS signals displayable by a panel in your laptop). Each type of display has its own signaling standard and protocol. Common standards for external connectors include VGA, DVI, HDMI and DisplayPort. Internal connectors are typically LVDS or Embedded DisplayPort on laptops. Configuration for each type is unique, and may involve different internal data links, like SDVO or FDI in the Intel case. Depending on the output, converting the data into a signal may involve dithering (reducing the color range for a display that can’t handle the full range) or scaling (like making an 800x600 mode stretch to fill the screen on a display whose native size is 1024x768).


So what does all this mean for someone trying to debug an issue when something goes wrong? In short, it means there are a lot of places to look for problems. However, with some knowledge of the way things work, one can narrow things down relatively quickly.

Source formats (aka “my display looks like modern art")

Source format problems are often the easiest to debug, since all the data is flowing through the system correctly and showing up on your screen, it’s just being interpreted wrongly somewhere along the line. This can cause the data to look squished, have the wrong colors, or otherwise just look weird.

Normal display Bad stride

The first image (on the left if your browser is wide enough) is a picture of what the display ought to look like given the mode & image I provided. The second image illustrates a simple stride programming error. In this case I intentionally misprogrammed the stride to a bad value, and you can see the display plane is feeding the data incorrectly to the pipe, interpreting each line as something much shorter than it ought to be, resulting in the skewed looking appearance of the corrupted image.

No dithering

This next image illustrates a bad dither setting. The panel I’m testing with supports only 18 bit color, but the encoder can dither the pixel data to reduce artifacts related to the color conversion. You may need to view the full size image to see the effect compared to the normal image, but if you look closely, especially on the white gradient, you can notice some banding of colors. This effect is proportional to the size of the gradient; a desktop background with a nice, full screen gradient would look horribly banded without dithering for example.

Pipe problems (aka “My display is winking at me. A lot.")

Pipe problems can manifest themselves in many ways, most often as a blank display. However, another fairly common failure mode is a blinking or flashing display. This is often the result of a FIFO underrun in the pipe. As described above, the FIFO is the part of the pipe that contains the pixel data to be sent out to the encoder(s). It’s periodically re-filled from RAM, either when a specific amount of free space is available or a specific amount of data is left in the FIFO (the so-called “watermarks"). If the time it takes for the FIFO to read from memory when its watermark is reached is longer than the time it takes to stream that same amount of data out to the encoder(s), the FIFO may underflow, causing an interruption in the pixel data stream to the display. This can manifest itself is shaking or flicker of the image on the screen, as in this video.

Other problems can affect pipes as well though. On recent Intel display controllers, there are two pipe-like units to worry about: the pipe on the CPU and the transcoder on the PCH. The CPU pipe actually feeds an internal FDI (flexible display interface) link between the CPU and the PCH. The PCH transcoder receives this data and sends it to the configured encoder(s). If something is wrong with the CPU pipe and FDI configuration, the transcoder may receive bad data or no data at all, resulting in odd looking images, which may blink or flicker over time. Likewise, if the transcoder is misconfigured, it may interpret the data from the FDI link incorrectly, underrun or overrun, again leading to strange looking images.

Encoder problems

Finally, all sorts of things can go wrong where the bits hit the encoder(s). The encoder may not be powered on at all, which would probably put your display into power saving mode or turn it off altogether (this is usually fairly obvious :). Or it may be displaying something, but without another critical resource enabled, like a backlight (sometimes on laptop panels if you tilt your display just right you can see an image displayed, depending on the opacity of the case behind your screen). Or it could be enabled with the wrong parameters, in which case your display may come up but give you a message about bad timings (usually indicating a problem with the pipe configuration rather than the encoder itself), or indicate a failure of the link some other way. I had some pictures of a panel that was on but without a backlight vs a panel that was turned off entirely, but they didn’t turn out well enough to see the difference (indeed it’s sometimes difficult to see the difference even if you’re holding the machine yourself). In short, encoder problems can be tough, because they often hide other bugs occurring earlier in the pipeline (e.g. if your display isn’t coming up, how do you know if the timings are correct? what about the source format or internal link configuration?). Really each type of output merits an article in itself, describing its operation and potential pitfalls; I’ll save that for another time.


Hopefully the above helps you understand how display controllers work at a high level. I’ll try to follow this up next week with some information on the details of current Intel display controllers (found in CPUs with Intel HD graphics), and what we’re doing to improve the debug environment of the display portion of our driver.

Obviously I’ve left a lot of detail out (what about EDIDs? how does the driver communicate with the display to convey or retrieve auxiliary information? don’t HDMI and DisplayPort support audio too? how does that fit in?), I’ll try to go into more depth in future installments. If you have particular questions or would like to see something specific, feel free to comment here and I’ll take a look.


Linux Plumbers' Conference 2011 Call for Track Ideas

Filed under: Announcements [A] — jbarnes @ 12:06:25 pm

The Linux Plumbers’ Conference will take place in Santa Rosa, CA, September 7-9, 2011. The structure of the conference remains the same as in previous years; rather than focus on presenting completed work or problems and solutions, we encourage BoF and brainstorming sessions that gather experts from different parts of the Linux ecosystem to discuss
complex problems with implications for multiple systems, with an eye toward making progress and reaching consensus on their solutions. As in past years, we aim to foster a workshop like atmosphere, where once consensus is reached, attendees can spend time developing solutions in real-time, sharing experience and evaluating trade-offs.

The program committee is now looking for track proposals. The ideal track proposal is focused enough to allow progress and attract the right particpants, but also spans several components of the Linux technology
stack and a set of problems that can specifically benefit from face to face discussion between the various teams involved. Proposals should include a list of projects involved, key contributors to lead discussions, and an owner for the track. The track owner is responsible for facilitating and scheduling sessions and generally making sure the
track is successful (metrics for this will vary by track & discussion).

Track proposals should be sent to lpc-planning@linuxplumbersconf.org.

LPC 2011 Planning Committee

Contact the admin. Powered by  b2evolution Credits: Dave Shea | skin converting | multiblog | green power hosting | test