PCI #5 - Designing an experiment within an humanistic paradigm

My previous problem proposition (as a reminder: “What is the impact for both the community and the company of community management when advertising an IT product on Twitter?) is in my opinion not a suitable problem to set an experiment. This issue needs to be researched in an everyday life context and not in an environment I would design and set up myself. Of course this is debatable, but to gain some time I’ll examine another problem. This new problem is actually related to a situation I experienced during my studies:

How does the usage of videoconferencing affect the process of learning of master’s students in comparison with a physical classroom in the context of academic lectures?

First here is a short justification for the choices made. In order to have a bigger influence some Universities tend to merge together and thus can have several campuses with similar lectures being given simultaneously in different locations (e.g. the University of Eastern Finland with 3 campuses in a 150km radius), videoconferencing is a good way to merge these distant lectures into one. I would study master students as they still have academic lectures but are rather autonomous, which will probably lead to a more nuanced outcome than with bachelor students.

To study this problem I propose an experiment. This experiment would last for a whole semester in order to cope with the possibility of a significant adaptation time from the students. It would involve 3 groups of approximately 20 students (a normal sized class), speaking the same language (in order not to add any problems related to the language barrier or cultural adaptation to the equation) and from similar backgrounds.

I would study 3 different situations:

Classical lecture: The teacher gives the lecture in the classroom where the first group of students is situated.

Video transmitted lecture: The classical lectures would be transmitted live to the second group of students in a regular classroom situated elsewhere with a video and audio feedback to the primary lecture theoretically allowing conversation between the two groups.

Totally decentralized lecture: a second teacher would broadcast live lectures from his office with the same content to the 3^rd group of students through a software such as Adobe Connect. The 3^rd group of students can be wherever they want provided that they have access to a computer and an internet connection. The lecture would be both audio and video with the possibility to display and edit slides live, the feedback from students could be only written on a common chatroom.

Throughout the semester a researcher would be present in both classrooms and in the 2^nd teacher’s office and would keep a journal about the way they perceive the reactions from the participants and keep record of the unavoidable technical incidents. At the end of each lecture, each student would have a really short questionnaire about their feeling about the lecture. At the end of the semester each researcher would conduct individual semi-structured interviews with the students from their group. Some quantitative data about the success rate of the different groups of students on a common final exam could also be used but with caution as it is in my opinion more relevant to focus on the process itself than on the raw outcome in order to improve it. The semi structured interview would focus on the way students handled their difficulties, the quality of interactions within the class, and their opinion on the medium used.

This should give some quality data to study to answer the problem, but I would also include the quality of students’ feedback in their assessment for the lecture in order to motivate them to give valuable outcome.

To finish, I have to admit that my experiment isn’t perfect. Some ethical aspects of it could really be discussed as the very foundations of this experiment is to study the differences between three mediums of teaching and thus consider de facto that they don’t give the same chances to students. Also acquiring some data on students’ performances on exams is could be criticized, as well as taking their participation into account in their final assessment. 

In order to appreciate the outcome of this experiment it would be necessary to find some previous studies on the subject. I didn't took the time to research further on the subject, but some case studies about the usage of video conferencing for teaching an learning are available here : http://www.icbl.hw.ac.uk/ltdi/vcstudies/vcstudies-all.pdf (Videoconferencing for teaching and learning, 1999). It's important to note that the technics have greatly evolved since then and thus, these case studies are probably more relevant to compare with the study findings about the second group of students than the third one who would be using a more complex technology. It could also be interesting to compare the results between the second and the third groups of students in the light of Q. Zhang and C. P. Fulford's article about the effects of having a one way video connection between the students and teachers compared to a two way video connection (Two-way and one-way video: when is 'no difference' significant in distance education? , 1996), even though, once again, due to the age of the study, the setup in their experiment was probably quite archaic in comparison with what's available today.

PCI #4 - Reformulating the problem using the ten steps (or at least trying)

Problem:

What is the impact for both the community and the company of community management when advertising a IT product on Twitter ?

People are connected all the time, they check their Twitter timeline often several times a day. Community management is not advertising per say, instead it's small bits of information distilled into the people's timelines that can change the image of a product for potential consumers but really appears like any other tweet and seems as neutral as anything else. How is this perceived by the final user and by the company? Depending on the results the effectiveness of this method can be assessed but mostly I feel that the differences between companies' results could lead to new ways to approach this kind of advertising.

Methods:

Twitter is a fast media. Tweets are short and people are spending the minimum possible amount of time on each tweet, so a short online questionnaire is more suitable for followers. It should be straightforward to have a significant amount of participants, but no one should feel like they have to answer, to avoid people who would fill out the questionnaire without even thinking about it.
Then we should run interviews with some community manager to have a view on the outcome they expect and what is actually happening.
With these methods we could have a broad and not really detailled view on how users feel about community management (and it's possible to sort them in function of the companies they follow) and at the same time focus on why and with what expected and actual outcome a some specific companies are doing this.
There is most probably other good options to study this problem, I just can't find any. Confronting this with another approach from someone intersted in this problem would allow me to refine my view on the problem. I feel rather stuck at this point of the reflection, probably because of a one-sided approach.


This ten step method which presented is in my opinion really heavy to use as it is. I think it actually requires a fair level of competence in formulating research questions and conceptualizing researches to be able to formulate all the intuitive reflections which are behind the decisions we could make. It's really useful though to check if we're not missing a point and assess the relevance of the process but would be heavy to use as a guide through the project. I say we because I think this is way more useful in a group research than alone since it's more difficult to be coherent in the research process especially at the beginning as a group than alone, and those steps have the huge advantage to force the researchers to formulate every aspect of the research and see if all the expectations are the same.

PCI #2 - Litterature review and empirical facts

What is the meaning of embodied in the litterature review and empirical facts ?

Searching for material to support a research and an idea isn't fundamentally difficult, but being embodied in the literature review and empirical facts takes you further than this. The first thing isn't about any research, it's about understanding. You can't just stay in a comfort zone of you usual approach, you have to understand as much different approaches as possible on your subject. There is probably as much different angles of view on one subject as there is people studying it and if we want to research further on it, we first have to try those different angles. And then while reviewing literature and searching empirical facts, this ability to change the point of view must be used to fully understand the context of whatever material we are studying and thus fully understand the material itself. Furthermore, as a paper is written mostly for external readers, it’s important to identify what is actually relevant for others more than for yourself.

What is a fact for Orlikowski and Iacono ? (Mind Over Machine, chapter 3)

The fact that is stated this chapter's argument is that a level of artificial intelligence comparable to the human mind cannot be reached. Of course this chapter is a state of the art at one precise moment, the models of AI presented there have been surpassed by newer ones since then. Especially, the neuron-net models and connection based procedures are really close to the way of operation of internet which is nowadays and good way for AI to have access to knowledge and even to assess the relevance of this knowledge. But this fact, AI cannot reach the level of intelligence of a human mind, because of the limitations they present and mostly because we simply don't understand totally the human brain, remains true today. This is a fact. The empirical facts and experiments they present to corroborate this fact may not always be still relevant today, but the argument of the authors still holds despite it and this is what makes it a fact and not just a hypothesis. To come to their conclusion they studied the different approaches researchers have tried for the last 10 years and assessed their relevance. They made categories and searched for the limitations applying to each category. They used the conclusions of theses researches as facts to establish their point.

PCI #3 - Problem Formulation using Creswell's scheme

Problem: 
What is the impact of the community management for advertising a IT product on Twitter ?

Reflection:
This way of formumating a problem forces the researcher to formulate all the components of the problem. We can tell wether it is about the process which leads to a result that we study or the result of a situation, the research angle, the studied situation, the parties involved and the context. Though it could be also restrictive or lead to bad formulations. If for instance I wanted to study this impact for community management in general, specifying the place (which would be "on social medias") would be redundant since community management is by definition on social medias. This template appears to me more as a summary, a reminder of what should be in a problem than a rigid formulation of it.

PCI #1 - Self-assessment of PBL competence using the D&D scale

PBL Skill	Level of competence
	Novice	Advanced Beginner	Competent	Proficient	Expert
Identifying a self-chosen societally relevant problem		X
Engage in search for literature		X
Engage in search for empirical facts	X
Problem delineation		X
Empirical and theoretical inquiry			X
Experiments			X
Production				X
Writing up in a group of collaborating students, a project report answering, but also critically discussing the problem initially formulated			X

In my opinion, about researching on a problem, the most important but also the trickiest part is about outlining the problem, and base the research on relevant elements.

I'm a beginner at identifying a relevant problem since I have to stick to some rules and some patterns during the reflexion if I simply want to have an outcome. The use of intuition is simply not an option on this one, though I make use of all contextual elements that can ease the task.

I'm also only a beginner in the search of literature. I have some methodology about it and tend to stick to it, taking the context into account, in order to find relevant references. The goal will only be to find some literature to base my reflexion but nothing will be especially daring or advanced.

The facts are even more difficult to search for, as while for papers it is easy to define if a paper is often quoted or used and thus probably relevant overall, for empirical facts, judging their quality and relevance can be more difficult, and I lack the methodology for it, I don't even know the rules to follow to begin with.

The problem delineation poses a real stake as if a problem is relevant, then narrowing it without loosing important fields of reflexion can be quite tricky. I do have some methods for it and some past experiences, but I still don't feel confident at all doing this.

Once the problem is outlined and the material for reflexion acquired, the actual reflexion and experimentation to create some result is already less complicated for me.

About theoretical and especially empirical inquiry I can follow some methodology, I'll modify it according to the context and past experiences I got and get rid of the useless considerations to go to the point.

Here I understand the production part as the way to lead the research project. As I got already some important experience in managing projects, I know the basic methodology, but mostly totally adapt myself to the context and others and really try to improve the way to manage it in comparison with previous experiences.

Writing a report is also something in which I have both some methodology and some experience. It still remains something I'm not comfortable with but I can use both my knowledge and my experience and do it in the perspective of a particular context.

About experiments, I don't get the difference between this and “Empirical and theoritical enquiry” so what I said about the latter one applies to the first one.

It is at this point quite obvious that I am not really comfortable with every aspect of the PBL approach. Though, this choice of skills is in my opinion perfectly relevant for a research project, it pretty much covers every aspect of the research process. But I'd give it a second look if it was for a more practical project. I happen to be from a rather technical formation, and on one side this doesn't cover every aspect of a more technical project, on the other all the skills above are not necessarily relevant on those projects. Without going further into what adaptations could be made (as it could take countless hours), I'll just take the question at its root : when speaking of PBL, is it necessarily a research problem or can it be a more practical problem which could fit in a more technical approach ?