Survey Feature

recently I spend quite some time creating a survey and evaluating it. I figured I'd share some thoughts about it here, maybe that'll be useful to someone.
 

I recently spend quite a while on creating and evaluating a survey done for the UK Indymedia network. While having the feature available at all was pretty great, using it wasn’t always straight forward and some things would have helped. The survey concerned a planned aggregator and involved lots of non-technical minded people, which made it a challenge in itself. I’ll try and explain things properly, but please do ask if I’m not totally clear.

creating the survey

The general way it works with adding different question types and then adding the content is pretty nice and works well for me. One of my issues was that our survey go quite long and working on it involved a lot of scrolling up and down to add questions and put them on the right spot. It would have been nice to be able to structure the survey into chapters or sections, with subheadings etc. Ideally this could also allow one to have the tools for adding questions at the top of each section, which would help with all the scrolling (and my confusion)

One thing we struggled with was that we couldn’t include images or links or format the text of the questions. Creating a survey for non-technical people it would have helped to have space with each question to provide explanations of technical terms, include screenshots, or at least link to those explanations. This also goes for the introductory text, it would really help to be able to write an introduction using all the normal wiki formatting.

premissions

once we were done creating the survey, we wanted to share it with the UK network and local groups. One worry was that people accidentally edit the survey itself or something like that. The permissions are not so straight forward and we ended up to basically keep the survey in private ownership, only giving write permission to groups and the network. This is a bit difficult as it means the survey can’t be moved to collective ownership, which would be the ideal option. I don’t really know a solution for this, but maybe there could be a way to lock the survey for editing the questions at some point? With a survey, you want to not be able to edit the questions once people have started to fill it in, not only to avoid abuse, but also confusion and accidents.

One problem we ran into with permissions, giving all the participants write but not coordinator privileges is that participants ended up not being able to edit their own responses. To do this they had to be given coordinator privileges.

Maybe there is a better solution for this and we haven’t gotten our heads around the permissions system?

evaluation

this bit was a little frustrating. This is partly due to us not testing things properly before going for it. When starting on the evaluation we were only able to see one response to the whole survey at a time. I had hoped that there would be at least the option to see all responses for one question. Or for multiple choice questions, just see how many responses went to which option. The evaluation ended up a lot more work than expected, as we had to go through every single response and then copy the results to a separate document. This wasn’t made easier through the way of displaying only the chosen option for the multiple choice ones, which meant we had to double check more.

I don’t know if we were just being thick, but there didn’t seem to be any more convenient way of evaluating the survey? Also, if there’s a deadline on a survey, it might be good to be able to close it for responses (and editing own responses) somehow. I guess that could happen by changing everyone’s permissions to view instead of write or something.

So my overall take is that a survey feature could be really helpful, especially integrated in a working platform like crabgrass. However using the survey at the moment brings some unexpected challenges. If you have any questions about what I’ve written, please get back to me. I hope some of my feedback is a bit useful. I haven’t included links to our survey, as it’s currently not public, and it seems a lot of work to get consent from all participants to publish their responses. (Is there a way to make the survey public, without publishing the list of participants and their responses?)

another nice function on crabgrass, that has great potential, but is still in the making. Cheers to everyone who has been working on this!

 
 

Nice feedback thanks a lot.!

 
 

great stuff, surveys have a lot of potential for sure and another development pass at them would go a long way. Hopefully we can tackle this soon. Your feedback helps a ton.

 
 

One thought for the evaluation I had, didn’t include it above because I’m not sure if it makes sense, but for the multiple choice questions, how about just using approval polls? Or including and approval poll type of question in it? That would give a few more options to the participants, and could result in a clearer picture, as it allows people to express that they are against an option as well as for it.

 
   

People of OSE Europe see big potential in using the crabgrass survey instead of limesurvey. Open Source Ecology is a worldwide network to develop a fully documented Global Village Construction Set.

So far for team culturing they used a wiki template whith questions that users can copy to their accounts page.

I suggested them, to use crabgrass for this and created a survey here.

First feedback was:

  • If you add some introducing hints, the layout becomes very dense as it is not possible to add line breaks or links, as markup is deactivated for the introduction text.
  • Some questions had been longer than the allowed text length. As links only can be added in plain text this was a limitation to explain the whole context of the question.
  • People shall be able to answer questions without the need to register.
  • An Option to export the answers in plain text or wiki markup would be very helpful.
    (see full conversation)

As azul pointed out lately “We hope to be ready to move we.riseup.net to 0.6 in spring 2012” it would be great if this will be incorporated.