The dual aspect of Drupal forms and what this means for your AHAH callback

Over the last week or two I've spent a lot of time on an aspect of my Quick Tabs module that I am certain none of its users will care a hoot about. It wasn't a case of adding a new feature or fixing a bug or even improving usability, but a question of, to put it succinctly, cutting down on its evilness. The admin form for creating and editing Quick Tabs blocks (where you choose either a block or a view for each tab) had a serious amount of ahah functionality: click a button to instantly add a new tab, click a button to instantly remove one of your tabs, select a view for your tab and have the view display drop-down be instantly populated with the correct options for that view. It was pretty user-friendly; there was just one problem: it flew in the face of Form API best practices.

When it comes to AHAH forms, there's the JavaScript side of things - where a behaviour is attached to, say, the onclick event of a button and new content is retrieved and inserted into the DOM - and then there's the PHP side of things, the AHAH callback where you rebuild your form with new or altered elements. Now, my understanding of FAPI voo-doo had been poor to begin with - I had competely copied poll.module for this side of things - and so when chx told me there was bad stuff going on in my AHAH callback and tried to point me in the direction of FAPI righteousness, I was pretty lost. Thankfully, he took the time to go through it with me, discovering that poll module was flawed in the same way, and though I didn't attain enlightenent straight away, the murk did eventually clear, I cleaned up my AHAH callback, and I think I have reached a stage where I can explain the crux of the problem to others.

You see, you need to think of your module's form (or any Drupal form) as being essentially dual in nature: there's the "material" form - what you actually see rendered on the page - and then there's the "spiritual" form, if you will, the form that exists out there in the ether (aka the form cache :-P). The material form and the spiritual form need to be representative of each other at all times. In fact, the material form should always be a "physical" manifestation of the spiritual form. If either gets altered without the other being altered accordingly, bad things will happen - at best, your form won't work properly, at worst, you will open the gates to FAPI Hell. For example, if you successfully add new elements to the spiritual form, so that the version of the form held in the cache now contains 4 poll choice textfield elements, but the material form, the form rendered on the page, still only has three poll choice textfield elements, then when you hit submit it will give you a validation error to the effect that you have left the fourth field blank, assuming such validation has been applied. On the other hand, say for example you have a dependent dropdown, where you populate the options in one dropdown based on a selection in another, if those new options don't exist in the spiritual form, you will get an error to the effect that "an illegal choice has been detected" when you submit. There are good and bad ways around this latter problem.

One way, the way many modules up to now, including my own, were doing it was to retrieve the form from the cache, tack on the new element, re-save it to the cache, rebuild the form and render the altered portion. This is illustrated below:

<?php
function myform_ahah() {
 
$delta = count($_POST['addable_elements']);

 
// Build our new form element.
 
$form_element = _create_new_element($delta);

 
// Build the new form.
 
$form_state = array('submitted' => FALSE);
 
$form_build_id = $_POST['form_build_id'];
 
// Add the new element to the stored form. Without adding the element to the
  // form, Drupal is not aware of this new element's existence and will not
  // process it. We retreive the cached form, add the element, and resave.
 
if (!$form = form_get_cache($form_build_id, $form_state)) {
    exit();
  }
 
$form['my_ahah_wrapper']['addable_elements'][$delta] = $form_element;
 
form_set_cache($form_build_id, $form, $form_state);
 
$form += array(
   
'#post' => $_POST,
   
'#programmed' => FALSE,
  );

 
// Rebuild the form.
 
$form = form_builder('mymodule_form', $form, $form_state);

 
// Render the new output.
 
$form_portion = $form['my_ahah_wrapper']['addable_elements'];
  unset(
$form_portion['#prefix'], $form_portion['#suffix']); // Prevent duplicate wrappers.
 
$form_portion[$delta]['#attributes']['class'] = empty($form_portion[$delta]['#attributes']['class']) ? 'ahah-new-content' : $form_portion[$delta]['#attributes']['class'] .' ahah-new-content';
 
$output = theme('status_messages') . drupal_render($form_portion);

 
drupal_json(array('status' => TRUE, 'data' => $output));
}
?>

The main problem here is that in between the form being re-saved to the cache and it being rebuilt, it is further altered by the lines

  $form += array(
    '#post' => $_POST,
    '#programmed' => FALSE,
  );

$_POST should not be used at all in the rebuilding of the form. Everything should come from $form_state, so that the rendered output is still a "physical manifestation", as I put it earlier, of the form in the cache.

In order to achieve this, a couple of things need to be done differently, the ahah callback needs to be altered - but this actually involves mostly removing code; and the function that generates the form needs to be structured so as to react to the contents of $form_state (when building the form it first checks $form_state for previously submitted information, then checks if it is receiving information from the database, then finally if it is not receiving information from anywhere, it renders a brand new clean form). The ahah callback then looks like this:

<?php
function myform_ahah() {
 
$form_state = array('storage' => NULL, 'submitted' => FALSE);
 
$form_build_id = $_POST['form_build_id'];
 
$form = form_get_cache($form_build_id, $form_state);
 
$args = $form['#parameters'];
 
$form_id = array_shift($args);
 
$form['#post'] = $_POST;
 
$form['#redirect'] = FALSE;
 
$form['#programmed'] = FALSE;
 
$form_state['post'] = $_POST;
 
drupal_process_form($form_id, $form, $form_state);
 
$form = drupal_rebuild_form($form_id, $form_state, $args, $form_build_id);
 
$form_portion = $form['my_ahah_wrapper']['addable_elements'];
  unset(
$form_portion['#prefix'], $form_portion['#suffix']); // Prevent duplicate wrappers.
 
$output = theme('status_messages') . drupal_render($form_portion);
 
drupal_json(array('status' => TRUE, 'data' => $output));
}
?>

The form is retrieved from the cache, processed, and rebuilt. During rebuilding, $_POST gets destroyed and the form is re-saved to the cache. You can see that there is no room for alterations to the rendered form that aren't mirrored in the cached form - so, then, where do changes to the form take place? Well, changes happen not in the ahah callback but in the submit handler for the element that triggered the callback. Your form is going to get completely rebuilt, so in the submit handler, which gets called during drupal_process_form, you can specify exactly how you want it rebuilt. Here's my submit handler for the "Add tab" button in quicktabs:

<?php
function qt_more_tabs_submit($form, &$form_state) {
  unset(
$form_state['submit_handlers']);
 
form_execute_handlers('submit', $form, $form_state);
 
$quicktabs = $form_state['values'];
 
$form_state['quicktabs'] = $quicktabs;
 
$form_state['rebuild'] = TRUE;
  if (
$form_state['values']['tabs_more']) {
   
$form_state['qt_count'] = count($form_state['values']['tabs']) + 1;
  }
  return
$quicktabs;
}
?>

There are two important steps here: we're passing all the form values (what has been entered on the form) to $form_state['quicktabs'] so that when the form-generating function is called, it will check here first to see if there are already values available for the elements it's building; we are also incrementing the number of tabs by 1 and passing this number in $form_state['qt_count'], which is where we tell our form function to check first for the number of tabs to build. [Note: when I'm talking about tabs here I'm referring to sets of elements, where each set corresponds to a tab of content in a final Quick Tabs block - not to be confused with the myriad other meanings of the word in Drupal]. With those in place we know that the entire form can be rebuilt from scratch, will be rendered faithfully on the page, and will contain everything we need it to contain, new elements and all.

Here is the submit handler for the views dropdown on the quicktabs admin form:

<?php
function qt_get_displays_submit($form, &$form_state) {
  unset(
$form_state['submit_handlers']);
 
form_execute_handlers('submit', $form, $form_state);
 
$quicktabs = $form_state['values'];
 
$form_state['quicktabs'] = $quicktabs;
 
$form_state['rebuild'] = TRUE;
  return
$quicktabs;
}
?>

Notice that it doesn't seem to make any change to the form at all - the form is simply going to be rebuilt but with a different selected value for this dropdown (coming from $form_state), and that will change the options in the display dropdown. The views dropdown uses the very same ahah callback function as the "Add tab" button, as does the "Remove tab" button - they only differ in their submit handlers. The code is therefore cleaner, easier to maintain, and most important of all - more secure.

So, despite its dual nature, with its "material" side and its "spiritual" side, the Drupal form can attain perfect oneness when this approach is taken :-)

To see the full Quick Tabs code, visit http://drupal.org/project/quicktabs and download the latest dev snapshot of the 6.x-1.x branch (2.x branch to be updated shortly). To see the form in action, click here. You will need to change the tab type to "View" in order to see the views display dropdown working.

Further Reading:
Doing AHAH Correctly in Drupal 6 and beyond

just what I needed

Thanks for going through this. I have been looking for a solution that is tab based for some time now. I have never been able to implement this in a fashion that pleases me. However, this is just what I was looking for. My site for an ice machines company will make good use of this method. I really appreciate you sharing this.

I am wondering whether I am

I am wondering whether I am missing something, but it seems that the version available for Drupal 6 does not allow for selecting views, only blocks. I can still pass on views as blocks, but there is no option for selecting arguments or limiting the number of entries in the tab as there is in the drupal 5 version.
thesis papers

Thanks for taking the time to

Thanks for taking the time to go through this and illuminate where I was off base. I figured I was, but it is good to know where. A few threaded responses and I'll post the code changes once I complete them back to the post.

That's because you use:
$form['files']['#options'][] = $file;
See the PHP manual - "Creating/modifying with square bracket syntax" for an explanation of $array[].

Ok good point. I should have caught that one. One of those to many things going on, not paying enough attention to things. Originally I tried $form['files']['#options'] = $files but I think other issues (namely the theming) got int the way. Sigh.

$form['files'][$file]['name'] = array('#type' => 'markup', '#value' => basename($file))
Filenames are not HTML and they need to be escaped with check_plain before you mix them into an HTML context.

Huh. I am sort of surprised by this. Not that I don't agree with you that check_plain() is good practice, but wouldn't the Drupal upload process sanitize things already?

For more information read katbailey's excellent microsoft certification...

Thanks for this link. For what ever reason, I did not find it.

Matt John

Input forms do not need to

Input forms do not need to use a theme (the parts are themed, but the whole form is not always run though a theme function). Depending on how you want it different you can use css to change the look or for "bigger" changes you can write a function the implements the form_alter hook and plays with the form that way.

While there is not always a theme function, the form API will call one if present. If the form is named ABC it will call theme('ABC', ...) and by default this will call theme_ABC(...). With phptemplate ou can add code to handle this as phptemplate_ABC(...) and then provide an ABC.tpl.php file to do the theming. In the case of comments are think the form name you want is 'comment_form'. Look here and search for theme_test_page() for a example of theming the form and here for how to override other theme functions.

Matt John
------------
ccna wireless

Amazing but...

The views dropdown uses the very same ahah callback function as the "Add tab" button, as does the "Remove tab" button - they only differ in their submit handlers. The code is therefore cleaner, easier to maintain, and most important of all - more secure.

Where the hell is this located, I have been searching for like 15 minutes? I feel so damn stupid.

Fuel | Energy

When coding, I use

When coding, I use drupal_set_message($var) to see the contents of my variables and it helps me a lot.
my problem is that drupal_set_message doesn't work in "AHAH callback functions" (functions that ends with _js).
How can I see the content of my varibles in such functions?

High School Diploma | Online accredited High Schools | Earn Diploma

Install the devel module and

Install the devel module and use the dsm() funciton. Then if you refresh any page after loading an ahah function, the dsm output will be displayed for you.

I sometimes use...

a textarea for debugging and i set the #value to print_r($my_var, TRUE)

<?php
  $my_var
//whatever variable i'm dumping
 
$form['my_debug'] = array(
   
'#type' => 'textarea',
   
'#title' => 'debug',
   
'#value' => print_r($my_var, TRUE),
  );
?>

it just makes life easier when working with larger form elements and arrays like the $form and $form_state, and that works amazingly especially, when working w/ AHAH.

It makes sense, but it also

It makes sense, but it also does seem to do a lot of unnecessary work just to add a new form field. Also, I would like to skip validation when dynamically adding new fields.
I like the 'old way' better.

Great post. If only all FAPI

Great post. If only all FAPI documentation were equally thorough! :) The code above works fine for me on my first iteration of AHAH but when clicking a button that was added via the first AHAH callback, I end up with an empty $_POST, and hence no form_build_id to get the form from the cache. Have you experienced this problem at all? Any thoughts/advice appreciated.

I have run into a similar problem

Hi mistresskim and katbailey,

I've been using 'non-best-practices' AHAH to make my form go. I wrote the page so it has 2 forms in it. The first is the data form, that contains data in a FAPI table, complete with little AHAH image buttons. The second form contains mini-forms in field specific divs, each with it's own buttons that fire off a 2nd tier of bad-practice AHAH to save the changes to the database, and rewrite elements of the 1st form, that display the data. For nice look and feel, I stuff the 2nd form into a div and run jQuery UI's dialog on it.

But, after reading all about 'best-practices' in AHAH I got all excited, because it looked like it would cut down on my code and simplify.

I originally had the problem that ahah wasn't attaching, but then I applied this patch:

http://drupal.org/files/issues/ahah_attach_settings_0.patch

Which gets settings set in your drupal_json() into the main Drupal javascript variable.

Then my second tier worked! woot! ... except, if I re-started the cycle again, first click then second click, my AHAH wasn't updating, any second ahah attaching was ignored.

I thought I'd figured it out when I decided to use my #submit function to find out which 'id' a particular new AHAH part was for, and I gave it a new $form entry (by 'id') and new ahah, but this lead directly to your problem, so my 2nd tier ahahs are there I click them, and like kim, I get an empty $_POST.

My next attempt at a solution was to create ahah 'responder' parts in the form from the very beginning, and use jquery to hide them. So they're all there from the beginning, just hidden, and I would show them by firing off jquery in my first tier ahah responses (which use jquery to show the appropriate controls, (which were of course rerendered by the #submit callback of the 1st tier ahah, to contain the data they needed).

I thought this would handle the problem by obviating any need for readding AHAH, however this too was a complete and utter failure. My 1st AHAH fires correctly, my jQuery fires displaying the correct 'second tier AHAH', but clicking the second tier ahah kills it.

At this point, I view Drupal 6, FAPI #ahah 'best-practices' in the same light as campaign promises and infomercials: Things that appear a whole lot better than they are.

I bought into this hollow promise and burned a week on this, (and my boss is probably gonna burn me back when I have no results to show). And it still can't add a button to a form.

I think we need an FAPI #ahah guru to show us how it's done.

Thanks

Hi Katherine,

Thanks for the nice writeup of AHAH functionality in Forms.

Developing the Dynamic display block module I also used some AHAH Functionality in the admin pages for different form content depending on selected options. We are now developing the next release of the module and have a lot more dependencies on selected options and I didn't like the way using the AHAH functionality used in modules like POLL and some other modules and what is used in views is at the moment to difficult for me. Also looked at the AHAH Helper module which looks like a great help in developing AHAH functionality although I have problems with setting different default values with this module, and it is not much maintained. I think your writeup will help a lot.

Also liked your writeups about JQuery functionality in Drupal. Used it also in the Dynamic display block module.

Thanks again, great help

Philip

Drupal...

I've been enhancing QR Codes or also known as Quick Response Codes. I guess Drupal is a big help in what I'm doing right now. This would be one of my reference starting now. I'm looking forward to your next posts. Thanks a lot.

Is it possible to add another

Is it possible to add another AHAH element by firing an event on an element which is added using AHAH previously?

GED Online | home school

AHAH helper module is being maintained

It's being maintained, but I haven't had the time to handle support requests. Sorry for that.

I am responding to bug reports, of which there has only been one since the initial release. So I think "not much maintained" is a slight overstatement :)

I'm replying here because I wanted to put the right nuance in place; I'd like to see more people using this module so I can get more feedback, because it is the direction core is headed.

P.S.: I'll try to reply to your support request this weekend.

AHAH Functionality

Hi Wim,

Thanks for your replies.

To make other people understand how to use your module, will have a positive effect on how many users will actually use your module and will make your module better.

I think your module is a great help in making AHAH Forms.

Like this writeup from Katherine to make developers understand how to make AHAH forms is very helpfull. (It's difficult to find the right way in handling AHAH forms, if you look at other modules how they use AHAH, it can easily put you in the wrong direction. The same you can say for the use of jQuery with Drupal)

AHAH forms are very user friendly for the user of the form but till now very unfriendly for the developer of the forms.

Thanks to you it becomes easier for the developer also.

Thanks again,

Philip

Insane

Isn't it utterly insane that every single AHAH callback has to do this whole dance itself? Repeated code = bloated code. Given the simplicity of basic Form API use elsewhere (i.e. drupal_get_form + two callbacks), the effort required to add this supposedly 'easy' AHAH stuff is disproportionately large.

AHAH helper module

The AHAH helper module (written by myself, sponsored by Mollom) brings disaster relief here, it'll prevent you from entering FAPI Hell.

For example, all of this:

or example, if you successfully add new elements to the spiritual form, so that the version of the form held in the cache now contains 4 poll choice textfield elements, but the material form, the form rendered on the page, still only has three poll choice textfield elements, then when you hit submit it will give you a validation error to the effect that you have left the fourth field blank, assuming such validation has been applied. On the other hand, say for example you have a dependent dropdown, where you populate the options in one dropdown based on a selection in another, if those new options don't exist in the spiritual form, you will get an error to the effect that "an illegal choice has been detected" when you submit. There are good and bad ways around this latter problem.or example, if you successfully add new elements to the spiritual form, so that the version of the form held in the cache now contains 4 poll choice textfield elements, but the material form, the form rendered on the page, still only has three poll choice textfield elements, then when you hit submit it will give you a validation error to the effect that you have left the fourth field blank, assuming such validation has been applied. On the other hand, say for example you have a dependent dropdown, where you populate the options in one dropdown based on a selection in another, if those new options don't exist in the spiritual form, you will get an error to the effect that "an illegal choice has been detected" when you submit. There are good and bad ways around this latter problem.

Well, this would simply have been impossible if you'd use that module :) Read my blog post about it for details, and to find out why dynamic (AHAH-powered) forms in Drupal 6 are flawed.

This post is exactly what I

This post is exactly what I am interested. keep up the good work. we need more good statements. ccna

Hi Wim, I am of course

Hi Wim,
I am of course familiar with your AHAH helper module - I have actually re-used some of the code from it in Quick Tabs (with proper credit to you in the README file) - specifically the override of the ahah success function, because I need ahah behaviours attached to the new elements. However, having a dependency on your module just wasn't an option because it's just for the admin interface - AHAH helper should be in core ;-) Also, the technique outlined in my post above is different from yours as it calls drupal_process_form() and drupal_rebuild_form(), which is different from your approach which does:

  // Now, we cache the form structure so it can be retrieved later for
  // validation. If $form_state['storage'] is populated, we'll also cache
  // it so that it can be used to resume complex multi-step processes.
  form_set_cache($form_build_id, $form, $form_state);

  // Set POST data.
  $form['#post'] = $_POST;
  $form['#programmed'] = TRUE;

  // Build the form, so we can render it, or continue processing it (call
  // validate and/or submit handlers)
  $form = form_builder($form_id, $form, &$form_state);

Anyway, I know there has been some discussion about getting an ahah callback utility function into core, AHAH Helper module being the most obvious starting point, so I look forward to following its progress. Do you think it'll make it into D7?

Katherine

AHAH helper for core

My technique is not as much a technique as it is a hack. It's possible that there are cleaner ways to do this, but if there are, they're way too hard to find, and then FAPI itself is too blame. I still think $form['#cache'] (i.e. caching forms in the database) is useless and that FAPI itself is overly complex.

This is likely to make it into D7 core, since it seems the code freeze won't be even close in February 2009, which is when I'll have time again.

The real issue to tackle here is to make FAPI elegant (by which I mean the code should be much easier to understand). Would you be interested to help out in that area? :)