Radically Transparent Research – or Why Publish Before Peer Review?

I was reading the Gardner Writes, as I follow my colleague and friend’s thoughts from the other side of the globe with great interest and anticipation of the thinking he forces me to do. He’s been on a multipart series kick lately, probably to break up a long piece of discursive writing that formed the spine of a report he wrote for his home institution, and thereby make it more accessible and easier to digest.

He made an aside, referencing a blog post by Dave WIner about why he (Winer) writes in public. This resonated particularly at this moment for me as we’re in the midst of writing a grant proposal for a funding body in Australia, one thrust of which is how to break the cycle of the traditional linear research methodology. In engineering education, the domain for which this proposal is targeted, as in most engineering and scientific disciplines, the process can be described as:

Conceive — Design — Implement — Operate — Analyze — Disseminate

(That’s a modification of the engineering methodology some of you may be familiar with from the CDIO consortium)

The point of this is that you follow your experimental protocol, conceptualising the hypotheses derived from a theoretical framework you think informs the work, design the experiment, its methodology to collect data that might inform and or refute it (them), translate the experimental methodology into something that might actually allow you to do the work, then conduct the experiment to collect the data and try out the methods you think will inform you about the questions you posed, analyze it when the experiment is over to see what came out, and then, perhaps with the others involved, you write it up to share it with your colleagues. This is the journal publication process that can take from as little as three to four months, to upwards of 18 months to 2 years. All up the cycle that you’ve just engaged in is a measured in years.

Since all of your colleagues in the community are likewise going through this process to explore their own hypotheses the asynchronous and overlapping time lines for this work naturally leads to the sharing of results occurring through the stages of the cycle. In fact it’s more likely than not that some of this new knowledge is likely to have value to your work, indeed to potentially influence it, and, if you weren’t fixed to the methodology you’re following so that you can get results that are experimentally sound, you’d likely have changed something along the way to leverage the knowledge that you’ve just read and discussed with the authors via email or at that conference you both attend annually.

All up the two year period that it takes to devise, conduct and report on your work IS as much of a problem as any aspect of the work itself. If the point of all of this is to learn and improve the educational process, the chances of doing that meaningfully in our life times is low. After all, many of the outcomes from this cycle aren’t going to be particularly informative – two years to report that you really didn’t find anything significant this time around is a long time in work life of an academic, and even longer in the educational trajectories of our students.

Dave Winer described why he writes his blog saying,

I write to express myself, and to learn. Writing is a form of processing my ideas. When I tell a story verbally a few times, I’m ready to write it. After writing it, I understand the subject even better.

The connection to why he writes and why we publish is similar. But the problem is the timescale. Blogging is relatively rapid. The short cycle time helps us share our thoughts and forces us to consider the things we’re thinking about. And, it invites others to see, consider, and respond to what we’re thinking about.

I write to give people something to react to. So you think the iPhone was a winner from Day One. Great. Tell me why. Maybe I’ll change my mind. It’s happened more than once that a commenter here showed me another way of thinking about something and I eventually came around to their point of view. And even if I don’t change my mind, it’s helpful to understand how another person, given the same set of facts, can arrive at a different conclusion

That’s the discursive dialogue that we’re missing in science, but which has been developed and understood in the open source community for some time. That’s the community from which Winer comes, and the transparent sharing of thoughts and ideas is part of the culture, the method of open source development. And that’s what’s missing from the higher ed learning research community. Our livelihoods are tied to the articles we produce, to the impact factor of the journals in which they are published, the attribution of the ideas to ourselves as the original authors. We sacrifice the enormous potential the community could give us by holding fast to the belief that if we share it openly we’ll see our intellectual contribution devalued, lost, or worse, stolen, by someone else claiming the idea as their own. We worry about our ideas being ‘stolen’ while in reality the fact that we put them out there in the first place establishes our provenance.

Once up on a time, when the mechanism of production and sharing of ideas took huge amounts of capital, significant effort to create the production value we sought in quality work, and complex, costly, and time consuming mechanisms to distribute the work to our colleagues in via the journals trucked/mailed/shipped to libraries in higher ed institutions around the world, the concern about provenance of an idea was more meaningful. Recall the Darwin/Wallace conundrum that surrounded the first published comprehensive expression of the idea of evolution by natural selection to the Royal Society in London. (Not familiar with this extraordinary coincidence of paradigm shifting ideas co-occuring? It’s a fascinating story, and one that owes a tremendous amount to Charles Lyell. Darwin was resigned to be ‘forestalled’ in getting the idea of natural selection as the driving force that works on natural variation (mutation) out to the world as his work. The process by which this was addressed, the compromise that emerged and the thoughtful intervention and guidance of Lyell is a lesson in ethical conduct, friendship and skillful political savvy).

The grant we’re writing is in part an attempt to demonstrate that there is another, more promising way to conduct this enterprise. The open notebook science movement has been around for some years – we are NOT claiming novelty in this. We are simply trying to appropriate the methodology and apply it to the research on learning design in engineering education. In this we’re adapting the work done by others, the most recent of which I’ve been reading is from the blog of Mel Chua. Here she writes

Radical transparency refers to the cultural practices used by healthy open communities (Free/Libre and open source software, open content, and open hardware projects) to expose their work in as close to realtime as possible and in a way that makes it possible for others to freely and non-destructively experiment with it.

From Mel Chua: Hacker. Writer. Researcher. Teacher. Human jumper cable.

We have to take the collective wisdom of the community and carefully apply it more rapidly to improve learning design for higher education students. We don’t want to put at risk the students in our courses, as is often raised as a “show stopper” concern by those who think that unless there is incontrovertible evidence of improvement in teaching approaches that are different, it’s better to stay with what works. But does it? That is, does it work? How do you know? And is there really the risk being asserted? As a colleague and co-writer of this grant once said in a panel discussion with me about the change in his teaching using the ‘flipped classroom’ approach,

Students won’t let you ‘fail’. They will raise concerns early and loudly is something isn’t going well. Any teacher worth being front of the room will respond and change what they are doing to avoid the catastrophe. Things may not go as you planned, but they won’t end up damaging the students in the class because they, and a good teacher, won’t let that happen.

So where is the real risk? It’s in not being open and transparent to learn together.


About longpd

I'm a senior scholar at Georgetown University, in CNDLS, a technologist and lapsed evolutionary biologist with an incandescent passion for new modes of seeing and learning.
This entry was posted in Open_Research and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s