Reserch Methods lecture, part II

I am now more awake and will give this another go--at least until my son wakes up from his nap.

I just read the journal articles after the "experimental" section in the lecture.  I have to say, I'm not feeling any more confident about my ability to interpret numerical data.  The second article focused on reading scores and attitudes, and the data seemed pretty straightforward except that the researchers threw in an extra variable.  Rather than simply making the read-alouds the only difference in the two groups, they threw in extra English instruction.  Why did they do that?  They do acknowledge this difference in the article, but I still don't understand why they'd do that if they were trying to show a connection between read-alouds and independent reading.  Honestly, I find this annoying.

The other study, focused on user satisfaction at a University library before and after specific customer service training, seemed much more valid because they didn't throw in a bunch of extra variables.

Note to self:  when designing an experimental study (if I ever do that), don't create extra variables.  It just makes it more difficult to interpret the data and makes grad students annoyed.

Qualitative Research:  Theory generating rather than hypothesis-testing.  Consists of observing or interviewing subjects, field research.  Usually about a specific phenomenon or occurrence.  Observation, qualitative interview, focus groups.  Suggestion for observations:  take "sketch notes" (words and phrases), then go rewrite them in more detail asap.  Include observations and your interpretations of them (note which is which).   

Articles:  The Williamson article was so incredibly dry.  For whatever reason, it did not hold my interest at all.  The McKechnie article was more interesting, but left me with some questions.  For example, if all the researchers were mothers, might that have influenced how they viewed the reactions of the children?  And what purpose did collecting this data serve?  It was, however, a good glimpse into field observation in a very chaotic setting, and I appreciated the candid discussion of the many challenges they faced.

Content Analysis:  "unobtrusive" research--method of observing social behavior without affecting it.  "The study of recorded human communications, such as books, magazines, web pages, poems, newspapers, songs, paintings, speeches, letters, e-mail messages, bulletin board postings on the Internet, and laws."  2 categories: conceptual analysis and relational analysis. Conceptual analysis records the existence and frequency of concepts. Relational analysis examines the relationships between concepts.  Researchers create "concept codes" and record instances of each concept.  Pitfalls:  how you categorize concepts influences results of study, researchers must ensure concepts are clear to all coding.

From the Koufogiannakis article:  "For many library schools, a course in research methods is now mandatory. However, if students are not seeing the practical applicability of the course, the meaning of it may be lost in their practice as a librarian."

I think this is absolutely true, as shown by comments in class discussion already.  Of course, the reality is that even if students do see practical applicability, other obstacles (funding, lack of time) interfere.

I think librarianship research may suffer from the same disconnect as much educational research (although there is certainly a lot of that).  One of my frustrations as an educator is that we have this huge push for research-based practice, but the reality is that often we do not implement research-based practice because of funding issues.  What, then, was the point of doing the research in the first place?  If we want research-based practice, then we need to provide the funds, time, and resources to actually implement those practices.  Otherwise, it's hard to see the value in the research.  At a time when libraries are being closed, privatized, and cut back, I doubt the reasoning behind programming and collection decisions is grounded in research--it's based on budget.  As long as budget trumps research, there will probably be this problem of not seeing the 'practical applicability" of research.

Or maybe I'm just grouchy and tired.

Research Methods lecture

This lecture is a standard lecture for all sections of the course I'm taking.  Frankly, it seems pretty dry.  In an attempt to focus myself, I'm going to summarize the sections here.


Surveys:  3 components:  1) sample (representative of larger population), 2) standardized questionnaire (clear, unambiguous), 3) method (self-completed, telephone, in person, online, etc.) 

Guidelines for formulating good questions: 

• Consider the Form:  Will you use statements, questions, or both?  If questions are used, will they be open- or closed-ended?  Both types provide challenges.
• Questions must be clear and unambiguous.  They should also be relevant to the users' knowledge--in other words, the people taking your survey should know enough to understand your questions.
• Keep it short and clear.
• Avoid negative statements and bias statements ("don't you think").
• Provide clear instructions in the introductory comments (and perhaps before specific questions).
• Pre-test it and correct any ambiguities or biases.

Send a cover letter to introduce respondents to the survey topic and obtain informed consent.  Parts of the cover letter should include:

• brief statement informing respondent that study contains research and a simple description of the project,
• name and phone number of principle researcher,
• clear statement of research purpose,
• clear instructions on procedures to be followed,
• risks/benefits/costs to participants when applicable,
• confidentiality guidelines,
• study is voluntary,
• for minors, informed consent note for parents/guardians.

Experimental and Quasi-Experimental Design:   focus on examining the effect of a specific action.  Can be controlled (ie, in a "lab" with something being done to subjects) or natural (ie, reactions after an event, such as a presidential assassination or natural disaster). 

Key components of classical experiment:

• Independent and Dependent Variables.  Independent variable can be present or not present.  Researchers observe what happens when the Independent Variable is present or not.
• Pre-testing and Post-testing.  The dependent variable is measured before the independent variable is introduced (or removed) and after.
• Experimental and Control group.  Populations of each group should be similar.

 However, this can be difficult in social experiments, so we have Quasi-Experimental Design:

• Time series design:  looks at behaviors over time, such as speeding rates before and after a new traffic light is installed.
• Non-equivilant control group: control group is similar to experimental group, but not determined by random assignment.
• Multiple time series design:  looks at behaviors over time in several areas or with several groups, in effect creating a non-random control group.

I am falling asleep as I attempt to download the readings in the lecture, so I will continue another time.

Terminology and Concepts from Chapter 2

Chapter 2 of my text (Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 3rd ed., by John W. Creswell) focuses on literature review.

Creswell begins by focusing on the question of study topics.  The topic is, simply, the subject matter of a proposed study, written as a few words or a short phrase.  Researchers gain insight or focus their topic in the planning phase by:

• drafting a brief title (my study is about ___), which can be changed/refocused as the proposal or study progresses,
•  stating the topic as a brief question,
• considering "Can this be researched?"  Are there willing participants? Do you have the resources to collect & analyze the data?
• considering "Should this be researched?"  Does it add to the available research knowledge, duplicate past studies, lift up the voices of underrepresented groups/individuals, address social justice or transform the ideas/beliefs of the researcher? How does the project contribute to the literature?  Does it pay off for the researcher in terms of career goals?  Have you solicited opinions from others in your field?

Creswell then focuses more specifically on the Literature Review as a way to consider a topic and as part of the expected reporting of a study proposal, journal article, dissertation, etc.

The purposes of a literature review are to share with the reader the results of other related studies, to relate the study to the larger dialogue surrounding that topic in existing literature, and to fill in the gaps/extend past studies.  It provides a framework for establishing the importance of the proposed study, as well as a benchmark for comparing the results/findings.

Literature reviews are used in several ways.

In a study proposal, consult your advisor, but in general lit reviews are a brief summary (not yet fully developed) of about 20 pages OR an outline of topics/references for development into a "literature review" chapter (20-60 pgs written).

In a journal article, the lit review is an abbreviated form of that found in a thesis/dissertation in a "related literature" section after the introduction, threaded throughout the study, or at the end (depending upon the type of study and methods used).

For qualitative research, use the literature review in a manner consistent with the assumptions of the participant.  Literature reviews can set the stage (as part of or near the introduction), can be a separate section with theory discussion, or can be a final section that compares/contrasts results of the new study with past studies. Creswell suggests using literature sparingly at the beginning of studies to convey teh inductive design of Qualitative research--unless the design type requires substantial literary orientation.  Consider the most appropriate place (beginning, seperate section, end), making your decision based on the intended audience.

For quantitative research, literature reviews are more substantial and are found at the beginning of the study to provide direction for research or hypothesis.  They may be used to introduce a problem in a "related literature" or "review of literature" section at the beginning of a study.  The literature review can introduce a theory or describe a theory that will be used, or it can discuss why it is useful to examine a specific theory.  At the end of a study, the literature review may be revisited to compare results.  Creswell suggests using the literature review as the basis for advancing research or a hypothesis in the introduction or a separate section at the beginning, and revisiting the literature to compare findings at the end.  Researchers must decide if they will use an integrated, theoretical, or mithodological approach (see below).

For mixed methods research, the literature can be any of the above, depending upon the research strategies used.   Creswell suggests using a placement consistent with teh study type.

Literature reviews can be INTEGRATED (summarize broad themes in literature), THEORETICAL (focuses on extant theory), and METHODOLOGICAL (less used today, focuses on methods and definitions).

An incredibly helpful, practical section of ch. 2 includes a list of steps in conducting a literature review, summarized below.  I wish I'd read this/taken this course as an undergraduate, as it would have helped me both there as a student and as a high school English teacher.

1. Identify key words to use to find materials at an academic library.  They may emerge in identifying the topic or from preliminary readings.
2. Search catalog holdings for the key words.  Focus initially on journals and books related to the topic.  Then search computerized databases.
3. Locate about 50 reports of research, placing priority on journal articles and books because they are easy to locate.
4. Skim this initial group of findings.  Copy/keep those central to your topic.  At this point you're just looking for "is this relevant to my topic," not "what does this say specifically about my topic."
5. Create a literature map (this is brilliant)--a visual representation of the literature on your topic (more on this later).
6. Draft summaries of the most relevant articles, including references in APA (or the appropriate style).
7. Assemble the summaries into a literature review, organized by theme or concepts.  End the lit review with a summary of your study and how it adds to the literature.

The Literature Map:  There is a good example of this in the textbook, but the literature map is essentially a "mind map" of the literature in existence.  Creswell suggests starting with the general topic, then breaking it into three broad subtopics.  Place specific aspects of each subtopic, along with relevant citations from the literature reviewed, under each subtopic.  Include quantitative, qualitative, and mixed research in your lit review.  Then place a box representing your proposed study in the appropriate spot, indicating its relationship to the existing literature.  In this way, researchers can easily see how their study fits into (or does not fit into) the existing literature.  What is new or different about their topic? Why is it needed?  I love this idea.

More on database searches:  I have long wondered when, exactly, I would learn this type of thing in "library school."  I was happy to see it in this chapter.  Creswell gives information on several free and commercial databases and their uses/benefits.  He advises using a thesaurus of search terms/descriptors from each database (if available) to refine individual search terms.  Free databases included ERIC, Google Scholar, and PubMed.  Commercial databases included ProQuest (which accesses several databases in one spot), Sociological Abstracts, PsycINFO, and The Social Sciences Citation Index.  I can honestly say this is the first time in my education that any book or instructor has discussed what differentiates one database from the next.  I feel like the next time I research I will make a more informed choice regarding where to begin my database searches, and the descriptions on the SLIS library liason's page will have a whole new context for me.

Creswell's priority in selecting literature:  This hierarchy helps beginning researchers determine which types of literature are most/least desirable.  This is another list that would have been helpful earlier in my academic career and in my teaching career.

1. Start with broad synthesis, such as overviews in encyclopedias or literature summaries from journal abstracts (this is obviously for giving the researcher a base of knowledge, not for specific inclusion in a scholarly lit review).
2. Then, look at journal articles from respected peer-reviewed national journals that report research studies.
3. Books.
4. Recent conference papers.
5. dissertation abstracts (pay attention to quality)
6. WWW.  (pay attention to quality). Check online journals to see they have solid review boards.

An abstract is a brief review of the literature (short paragraph) summarizing major elements.  Abstracts of studies mention the problem, state the central purpose of the study, give information abotu the sample/population/subjects, review key results, and, in the case of methodological reviews, point out flaws in research methodology.  Abstracts of other literature mention the problem, identify central theme(s), discuss major conclusions, and, if methodological, mention flaws in reasoning,  logic, force of argument, etc. 

Creswell also mentions style manuals.  APA must be my new best friend.

Definition of Terms:  this may either be part of a lit review or separate from it.  Regardless of study type, the definition of terms section should:

• define terms that individuals outside the field of study might not understand.
• define terms when they first appear in the document.
• define terms in all sections of the research plan (title, problem statement, purpose statement, research questions/hypotheses, or objectives, literature review, theory base, and methods).
• define terms at a specific, operational level (as used here)
• use accepted language in research literature (not everyday language)
• definitions can define a common language word in the research context, pair a common term with a limitation, list a criterion, or define a term operationally
• One approach is a 2-3 page separate section.

Qualitative studies should delay definition of terms until they appear in the study.  Writers may pose tentative definitions before their entry into the field if necessary, but definitions should emerge as the study progresses.

Quantitative studies should include extensive definitions in the research proposal in a separate section at the beginning.

Mixed Methods studies vary depending up on the methods used in the study.

In Quantitative studies, structure literature reviews by first discussing literature focused on independent variables, then literature focused on dependent variables, then studies focused on the relationship between the two.

There is a lot to digest in this chapter, and being able to apply it to research of my own (or hypothetical research of my own) would be helpful.  I can see how becoming a good researcher requires putting on a different "hat" or mindset than the practitioner hat.  Which probably means now is a good time to go read the assigned article, which I think focuses on practitioners as researchers.

Terminology from Chapter 1.

It took me a while to wrap my brain around this chapter, both because I faced constant interruptions and because I had to adjust my thinking from a practical to theoretical view.  Once I did that (and locked my door), things did not seem nearly as complicated. 

Key concepts:

There are three over-arching elements of research:  1. the world view of the researcher/study, 2. the strategy of inquiry, and 3. research methods.

Four Worldviews: 

1. Postpositivist:  This seems the most "scientific" to me.  Postpositivists believe causes determine effects or outcomes.  They try to reduce ideas to a small set of "truths" to test, but they believe all research is flawed and you can never actually reach absolute truth.  Postpositivists develop a theory first, then test it, then revise the theory, then retest.  They are seeking truths while acknowledging the impossibility of finding it.

2.  Constructivisim:  This seems the most "natural" to me.  Constructivists believe that individuals make meaning based on their experience.  Therefor, meaning is subjective, varied, and multiple.  Researches look for complexity of views.  They ask broad, general questions, focus research on observation of discussions or interactions.  They also address the processes of interaction.  Their intent is to make sense of meanings; therefor, while they may begin with a general focus, they construct their theories based on their research results.

3.  Pragmatism:  I like the flexibility of pragmatism, since pragmatists do what works.  Pragmatists  focus on application in their research--they are looking for what solves problems or what works.  This approach draws from both qualitative and quantitative research methods, choosing those that work best for their research.  Research occurs in context, so that flexibility is important.  They also hold that "truth is what works at the time."

Interesting quotation from textbook:  "Pragmatists have believed in an external world independent of the mind as well as that lodged in the mind.  But they believe that we need to stop asking questions about reality and the laws of nature."  To me, this indicates that pragmatists feel the laws of nature or reality is pretty much fixed or beyond human control, so researching it further does not serve any true purpose.  They seem to have accepted this and chosen to focus on how human beings interact within those natural laws rather than questioning the laws themselves.  This just struck me as interesting.

Three Strategies of Inquiry:

1.  Quantitative:  Includes experimental designs (control group, etc.) and non-experimental, such as surveys.  There is an intent of generalizing data into findings/theory.  To be honest, this strategy makes my head hurt.  While I see the value in quantitative data in some cases, I have very little experience or expertise in analyzing this type of data and am far more interested in "what does this mean" than in the numbers themselves.  I may need to retrain my brain in this area.

2.  Qualitative.  Focuses on more "open ended" research strategies.  The text discussed five types. 

•  Ethnography refers to research done on an intact cultural group in their natural setting over a prolonged period, and research is conducted using observations and interviews.  This makes me think of Jane Goodall and the apes. 
• Grounded theory is so called because it is grounded in the views of the participants.  Researchers derive a general theory based on the participant views.  They collect data in multiple stages, comparing data with emerging categories and theoretical samplings.  In other words, they make sense of the information as it emerges rather than waiting until all data has been collected to develop theories.  Personally, I would find it very difficult to stop myself from drawing conclusions until all data had been collected.  I'd constantly be thinking "maybe this means" or "this is related to that because."  
• Case studies are in depth and occur over an extended period of time on one or more subject (the "seven" movies, that one study of nurses over decades).  
• Phenomenological Focuses on a small number of subjects bound by a common experience or phenomenon.  Interviews and observations are used over time.  Researchers set aside their own experiences.  Conclusions focus on patterns and relationships within collected data.
• Narrative involves the researches life and experiences as well as those of the subjects.  Researchers ask subjects to provide stories about their lives.  This information is retold by the researcher, who creates a collaborative narrative chronology.  This reminds me of documentary film making.

3.  Mixed Methods.  This strategy came about because researchers felt limited by a quantitative OR qualitative approach.   It allows researchers to use both research strategies/methods, using the two types of data to validate each other or focus/expand on findings.  There are three main types.

• Sequential, which seeks to elaborate or expand on findings from one method with another.  For example, a quantitative survey may shape open-ended questions in subsequent interviews.
• Concurrent, in which both types of data are collected at once.  Researchers attempt to converge or merge data to provide comprehensive analysis. 
• Transformative, which involves a "theoretical lens" as the overarching perspective within a design which includes both quantitative and qualitative data.  The lens provides the framework for topics, methods, and anticipated outcomes, and research could be sequential, concurrent, or both.  I am unclear on why this is its own category (other than being able to have both sequential and concurrent research).  Don't all studies involve a "theoretical lens"? 

At this point in the reading I felt like I had a pretty good grasp on these concepts.  The tables in the text were helpful in solidifying, clarifying, and differentiating these concepts.  Choosing which method to use depends largely on the research problem, the training of the researcher, and the intended audience for the research results.

A New Chapter

I am now going to be using this blog for a different course--LIBR 285: Research Methods in Library and Information Science.  Sounds exciting, no?  I actually do think it will be kind of exciting.  The course (I think) focuses on doing primary research, which I have no (or practically no) experience with.  In real-world terms, this translates in to me learning how I can research my own practice as a Teacher-Librarian (or even as a classroom teacher) and translate my findings into quantifiable data.  This is important to me because the reality of public education is that TL's have to justify their presence in schools.  If you can't show administrators and taxpayers that what you do impacts student learning, then you probably won't have a job for very long.  Like it or not, TL's have got to advocate for themselves.

Beyond advocacy, approaching my practice with a research mindset will no doubt improve my practice.  Research, or perhaps more accurately the analysis of that research, is by its very nature reflective.  I challenge anyone to fail to improve their performance when they are truly reflective about what it is they do on a daily basis.

For now, I know very little.  Perhaps I should complete some of the assigned readings and learn some more.

I hope this blog can continue to be a record of my growth during my quest for the TL credential and my MLIS.  Originally I'd planned it to be only for the one course (Instructional Design), but I'm kind of excited to use it for this class and maybe others in the future, whether that is assigned or not.  I guess I'm just nerdy that way. :)

Feeling Appreciated.

A. just sent me an email about our KBC. She called me her "dream librarian."

That's pretty awesome. :) Nice to feel like the work of teacher librarians is appreciated and that A. will be able to use what we did for her class.

KBC is UP!

Hooray for weekends--I built the KBC today. I'm pretty pleased with how it has turned out so far, and sent A. the link so she could look at it as well. Hopefully I'll get her feedback tomorrow and can make adjustments from there. It would be nice to be able to turn this in before class on Tuesday!

Working with A. has been really energizing. I'm excited that some of my hard work might actually be used with students next semester, even though this class will be over by then. I'm also interested to see how closely our lesson plan is followed or if A. ends up making changes to suit her style (or compensate for the fact that I won't actually be there). I hope she keeps the student learning blogs as a component because then I can read/comment on them from afar.