Robo-Trainer Presentation Style Too Stiff

There’s no question that there’s a current “buzz” among trainers around the use of technology, and particularly “social learning”, to facilitate, and make more efficient, learning in the workplace. Elliot Maisie is considered one of the leading experts on the topic of workplace learning in general, but also in the use of technology for that function.

Elliott Masie is an internationally recognized futurist, analyst, researcher and organizer on the critical topics of workforce learning, business collaboration and emerging technologies. He is the editor of Learning TRENDS by Elliott Masie, an Internet newsletter read by over 52,000 business executives worldwide, and a regular columnist in professional publications.

So it’s not odd that people look to his work to find out “what’s happening” and look to his “research” for guidance. Except, is it really research? Or is it a means, intentional or unintentional, to promote a biased agenda? You have to wonder about this if you look at the actual work. For example, there’s the Elliot Masie Survey on Social Learning which purports to take a quick pulse of learning colleagues around the globe on current implementations and plans for Social Learning.

While there is almost no information about the respondents of the survey, except that it was sent to subscribers of Masie’s Learning Trends, there’s NO information about the methodology used, non-response rates, or other details that are essential to interpreting the data.

What Can You Conclude From The Data?

Actually nothing. With biased sampling, poor item design (I’ll save you the explanations on that), no methodology or information about response rates, no demographics of the respondents and other issues, it’s garbage in, garbage out. The results are meaningless, and despite the fact that Masie does say it’s a quick pulse taking, one has to question the professional judgment involved in publishing survey results that are not only useless, but misleading as well.

The average reader of results like this will accept the conclusions at face value, since the commentary does not adequately explain the limitations of the data or the findings.

It’s standard procedure in research to speak to the limitations of a study, particularly when there is no explanation of the methodology, and when the key information to interpret the data is not presented (which can happen due to space limitations.

What’s Going On Here?

It’s quite possible that the people who carried out the survey are simply incompetent regarding survey design, surveying methodology, and research reporting. Few trainers, teachers and practitioners ARE competent in these areas. Or, it could be they don’t care about whether it’s accurate or not, or misleading or not. I don’t know, in this case. I am sure there is no conscious attempt to deceive.

Yet this is common. You have a person or group of people with an agenda that is biased and that has embraced a particular view of the topic they are “researching”. They begin as advocates for a particular tool, technology or approach, and THEN research it within the supposition that is of value.

Then the results are used to create an appearance that the particular tool is being embraced out there (even though it may not be), or that a large majority of people see it as valuable. Often the data used is sloppily collected, and interpretable, but people in advocacy roles tend to overlook the limitations.

The research on the MBTI is such a case, since so much of the research is undertaken by people already advocating its use. That, in itself, is not a problem if the research is published with full details, and the limitations of the research are carefully explained. When that is not done, you have to question the degree to which the researchers are using proper professional and ethical judgment.

Sadly, this appears to be the case with people researching e-learning, and social learning. It is also a major reason why training is riddled with fads that come and go — faulty research used to create “buzz” and/or an appearance of efficacy (often survey results are used in this manner). Note: Informal learning is the 2009 topic.

A Few Messages

1. If you are going to do research, do it properly in the first place.
2. If you are going to do badly designed research with flawed methods, don’t publish it anywhere. It will make you look either incompetent or unethical.
3. If you do badly designed research, and publish it, then at least tell people why the research is meaningless. You may still look incompetent, but at least you’ll look like an honest incompetent with integrity.
4. Keep in mind that the information you publish, if presented as research findings, will be used by others to make decisions, or to influence others. Bad, misleading research is not benign.

Finally, for readers of  “research”, start using more critical thinking. If more readers knew the basics of research, both methodological and ethical, and called to task those people passing off meaningless data as meaningful, we might end up with a lot less junk research, and a lot better information on which to base decisions. And that means less money wasted, and more credibility for the training profession and the function of creating learning in the workplace.

Comments invited.