1242:
effort. What I do think we should do is something like this: when professors sign up for the spring semester, they should be notified that we may not have enough volunteers to adequately support every class, and we will notify them if we can't provide any ambassador support so that they can decide if they want to continue without that support. We should also ask them as part of the signup if they would be willing to work on improving an article in their own field as part of learning how their students will be engaging with
Knowledge (XXG). Any professor who answers "yes" to that question should be given top priority in getting support -- they should get ambassadors for their class who are prepared to work with the professor as well as the students. Finally, if we decided (e.g. from the questionnaires being sent out) that the right student/ambassador ratio is, say, 15:1, then we should identify the classes we will support and ask ambassadors to assign themselves to classes in that list, and keep a list of unsupported classes so that additional volunteers who come along mid-semester can find more classes to support.
364:
support to their classes, but in return we should be asking for them to participate, as part of the
Knowledge (XXG) editing community, in curating the articles in their specialities. Writing a good article or featured article on Knowledge (XXG) requires more than knowing a topic inside out; you have to understand what the experienced editors here understand about what makes a good encyclopedia article. We have that, and the academics have knowledge – and now, finally, thanks to the global education program, we have something they want – an innovative and productive way for them to teach their students. Any instructor who gets a couple of good articles under their belt will understand Knowledge (XXG) well enough that they will be a much better instructor for their students, and will in turn reduce the burden for the ambassadors supporting their class. We need more subject matter experts, and we need the education programs in the US and elsewhere to help us acquire them.
1392:. The Cyrus Cylinder article is about an artifact a few hundred years older than the Hoxne Hoard which has had the misfortune to become a symbol of Iranian nationalism, complete with pseudohistorical claims and outright falsifications. I've collaborated with a couple of academic experts to develop the article. However, although the article has been improved significantly our only reward has been to experience denunciations off-wiki by Iranian nationalists (some of whom are frankly a nasty bunch) and repeated episodes of Iranian Randies attempting to hijack the article and delete academic views that they dislike. Frustratingly, other editors have usually tended to avoid getting involved in resolving disputes, even though the article has nearly 100 watchers. The bottom line is that often the only thing protecting articles against Randies is an active and involved editorial community, and for specialist subjects this is all too often absent.
1292:
person. We really need to move up to the level of graduate student or those with academic or professional careers in the subject before their degree has much influence on the quality of their writing. Traditionally, Knowledge (XXG) contributors have been intrinsically motivated volunteers. These student assignments mark a shift in the editor population towards extrinsically motivated writers. There is a pressure to complete the assignment even if one is not proud of the result -- whereas a volunteer might give up or try something less ambitious. And not all the students will be any good. Previously, a dire essay would end up as so much paper in a drawer. Now such bad writing might harm an article read by thousands of people every day. Obviously, the opposite is also true: great students will get the chance to have their fine writing reach a worldwide audience. But these students aren't getting sufficient support.
1031:
for the core values, then that filters down to the class-work; indeed, I think that was the major reason for the failings with the India programme - the "drive from the top" was inadequate. I also fear that WMF is taking too much of a back-seat; as the programmes expand, "we" (the projects and/or WMF) need to provide the tools and support to energize the key programme participants; it seems that the programme leaders want the programme to be self-perpetuating, and whilst it is admirable to want academia to go forth and create their own programmes, it needs careful guidance, to keep things on track. It's a tremendous opportunity for the long-term future; it's a very worthy investment, but we cannot expect it to grow without sowing seeds.
1223:
instructors or classes are doing the best work, so we can try to understand why). Are the questionnaires that were put together for the instructors and students the way you plan to evaluate the quality of the students' and instructors' experience on
Knowledge (XXG)? E.g. did they get enough support from the ambassadors? Were there benefits from working on Knowledge (XXG)? What interactions did they have with the community and did this help them learn? If so, then I think that will be very helpful -- for example, if we universally get comments that the classes received enough help from their ambassadors, then it's clear I'm wrong about needed to reduce the number of classes we're supporting.
1485:
Retired profs have more time at their disposal, no such requirement to write, and — we can hope — an interest in passing on their knowledge to future generations. Logically the group to target, it would seem. The major impediments are
Knowledge (XXG)'s oft-times nasty internal culture and the lack of some reasonable approximation of WYSIWYG content-writing software. (We still expect our contributors to write code, albeit simple!) I think the Randy-in-Boise phenomenon is a fictional cartoon, the real problem relates to unrestrained POV warriors escaping discipline. Hopefully and presumably WMF is at work on the software issue. The internal culture issue is necessarily a work in progress.
254:, it's not even enough to have good sources; to do the job right you have to have a truly broad understanding of the literature – which sources are fundamental to the subject, which theories are regarded as mainstream, the ideas that need to be covered in the main articles, and what can be relegated to subarticles or ignored completely. In other words, there are many articles that a non-expert simply can't improve above a certain point. This doesn't appear to be a problem caused by poor editor retention. It's certainly true that we need to do a better job of keeping new editors engaged, but simply increasing editor participation won't automatically focus attention on underserved areas.
1522:
that they might want their field, which they clearly care about enough to study for many years of their lives, to be better represented in
Knowledge (XXG) articles. I am a mentor in the ambassador program (was involved on the Public Policy initiative last spring) and am also an academic. My experience with the students I mentored, in the main, was like the vast majority of students I teach. Student participate as part of a means to an end (passing a class, or getting a degree) -- only a very small number are intellectually interested in the topic, or in contributing to a growing body of knowledge called Knowledge (XXG). Getting more academics productively contributing to this
1115:, or if not themselves, then their research group, institution, results bearing their name, etc. Two, I've found them often a bad judge of "neutrality", especially in areas they are too close to; as in politics, someone themselves engaged in a dispute is not the best person to write about it. And, many academics are engaged in acrimonious debates within their field, or between their field and others, and they can have very different views of what "neutral" is. Of course, some of these problems can be overcome or mitigated; when I try to recruit new Knowledge (XXG) editors from academia, I usually suggest they start with articles broadly in their area but not their
331:– too rare to justify the significant resources that the WMF and the community are putting into the program. It could also be argued that the use of Knowledge (XXG) as a teaching tool is a contribution to the public good, and a demonstration of the value of Knowledge (XXG). Both these things are true, but neither point should be allowed to preempt the primary goal of improving our encyclopedic content. I believe the USEP's significance lies instead in the fact that it is the first time that there has been an attractive way for academics to engage with Knowledge (XXG). It is the academics who can make the investment in the USEP worthwhile for the encyclopedia.
368:
816:, for instance, hasn't been removed by a mob of ill-informed editors. On the whole on Knowledge (XXG), good contributions drive out bad. The same is true of good contributions from academics, so long as the academics are aware that the social norms of Knowledge (XXG) are different from those of academia (Note that academia isn't exactly a stranger to long-running flamewars, but theirs tend to be conducted at conferences and in learned journals!). The main challenge is getting busy academics to understand that contributing to Knowledge (XXG) extends their own personal reach and impact.
348:, and it is easy to find editors complaining about individual students or classes making a mess of articles this past semester. This sort of impact on the community is only worthwhile if we are gaining significantly from the program – and the biggest return on investment we can get here is the academics, not the students. In fact, the current focus on growth is actively harmful – as supervision by ambassadors becomes more and more stretched, we will have thousands of students contributing to Knowledge (XXG) without truly understanding it. The burden of coping with this will fall on
311:, in which academics in public policy programs were recruited to use Knowledge (XXG) as a teaching tool – they asked their students to improve Knowledge (XXG) articles in the area of public policy. Existing editors were recruited to help the students, and the result was a success – the students added nearly nine million bytes of data to the encyclopedia, and a statistical analysis of the quality of the added material showed that the average article worked on improved dramatically. The next step was to expand beyond public policy: the
193:
267:
fact in question is something they regard as so basic as to need no evidence. On the other hand, a dispute may arise not because a reviewing editor ignores credentials but because they lack expertise: broad statements can be hard to source, and it may be that only an expert can judge accurately whether a source fully supports a given assertion. These are real concerns, but despite this, some academic groups are already beginning to take the initiative in improving these topics. The
1337:
all. So I see the disconnect between academic and WP writing from the WP point of view. I can only guess at some of the reasons for the mistakes, but I'm sure some of my guesses would be right. There are many people on WP who are in academia or have much more experience of it than me who could help make such a guideline. If someone wants to say where to put the essay, then I could dump some thoughts and ideas on the talk page and we can see what happens from there. --
961:
sourcing and referencing. I recall during my first week as an undergraduate being dragged around the library and having drummed into me the importance of reliable sourcing. It may be that the failed projects have focused on the wrong things. There's no reason to believe, for instance, that a student in India, Pakistan, or anywhere else has any special knowledge of any social sciences topic, but they ought to be able to learn from trying to write an article on one.
1269:
may not understand the subject—wiki editing—well enough to teach it themselves. Instructors would be well advised to gain significant on-wiki experience, have their work and their community interaction critiqued by an
Ambassador, and review case studies of successful and unsuccessful class exercises. Anyone can edit, but not everyone can teach wiki editing without preparation—professors with a sense of professional responsibility should already know this.
356:, and the editors who already have the students' articles on their watchlists. Even initiatives such as the Association for Psychological Science's call to action are likely to end up using the USEP approach, rather than having the professors edit directly, partly because the template for using Knowledge (XXG) in the classroom exists and is being widely publicized. There are other education programs for the English-language Knowledge (XXG), such as the
1138:
from ambassadors) and will leave. That's the right outcome in that case. If it turns out that academics who are willing to try editing
Knowledge (XXG) and capable of doing it well are very rare, then my proposed approach is no more efficient at using ambassador resources than the existing approach. However, I know enough academics who do edit Knowledge (XXG) well to think that there's no need to be too pessimistic on that score.
682:
1302:
traffic is so low one could go on holiday for a month and see no edits while away. Writing for WP is a unique challenge that requires a major shift in writing style for some. Some just don't get it, and then blame WP when issues arise. I think it would be valuable to write an informal guideline or essay to help those used to academic writing adjust to what is expected on WP. That would help both the profs and the students.
120:
110:
862:. When we're talking millions of dollars and hundreds of users being thrown at something, you'd think that troubleshooting would take place well ahead of time. If they did so, they certainly did a piss poor job of doing so. We need to put a cap on the programs: they're not allowed to get bigger until Knowledge (XXG) can support them. As for the India outreach program, well some programs just need to be cut entirely.
1468:. Content is, after all, the only reason an academic would want to associate themselves with this somewhat seedy website. Leave the user accounts, administrator roles, and other advanced-levels of Wikipedium to the wikiholics, drama-mongers, and gamers. Keeping the expectations low is the best way to avoid problems. When an academic expects to be treated badly, they aren't as upset when it inevitably happens.
280:
1218:
ambassador, which seems much too high to me. The second question is how we are planning to measure quality. There are two kinds of relevant quality, I think: quality of work and quality of teaching (by which I mean the quality of the student/instructor experience). The quality of the work done is an issue to the community, and I'd like to see some version of the PPI metric used. The
33:
130:
531:
acknowledgement that referencing requirements should be tuned to the breadth of a topic -- the larger a topic, the lesser the need for detailed page-referencing of every line, and the greater the need for reviewers with enough expertise to have a good sense of whether an article is accurate and comprehensive without having to consult sources regarding every line.
90:
1183:
to support our students better next term. One of the USEP's major goals for next semester is to improve our support and resources for students, professors, the community, etc. For this reason, we've aimed to maintain our current number of classes that are a part of the program (in other words: not grow!) so we can focus on deepening our support structures.
140:
215:, and browsing the encyclopedia frequently led you to articles so weak they could be improved in a few seconds without sources, just by typing in information you knew off the top of your head. There's still an enormous amount of work to do, but it's now rarer to randomly stumble across articles that can be improved without significant research.
976:
or it's talk page. The kick in it for the academic would be that the academics would have an easy outlet for their views, and would get credited for it. We would have more material and sources. They may be allowed to write new articles from their own work, with the caveat that they should not insist on keeping the WP version to their liking.
100:
1388:, which Rich Farnborough mentioned above, about another British Museum item. The Hoxne Hoard article was written by a group of editors (including me) as a highly focused collaboration with experts from the British Museum. It was successful and effective not just because there was a good bunch of people involved with it, but also
1521:
I very strongly support the ideas expressed in this essay. I support quality over quantity as an emphasis, and I support the idea of principle focus on academics and encouraging/assisting them to wade into the
Knowledge (XXG) edit space, difficult as that can be for some of them, for the main reason
1435:
Good essay, Mike. You explained the current situation with background and made specific recommendations, including how to measure the success of those recommendations. There are several methods for dealing with the "Randy" issue, such as by putting all featured articles on sighted revisions, but WP
1268:
Good essay. Trying to stimulate growth in a community that needs to grow organically is a real balancing act. I have long thought that the
Ambassadors' skills would be better invested in teaching interested instructors about the wiki way, rather than serving as teaching assistants for instructors who
1137:
These are all good points, but
Knowledge (XXG) has fairly strong selection mechanisms that I think will filter out some of these problems automatically -- that is, an academic who insists on self-promotion is eventually likely to get frustrated with the opposition from other editors (not specifically
895:
was their attempt to lay out infrastructure, and they did a pretty good job with it. The USEP is not a major problem yet, though it is creaking a little; my argument is that we're heading in the wrong direction, not that the program is already broken. I agree in principle that the right approach is
275:
included a "Call to Duty" in their November 2011 newsletter, with two goals: "first, to improve the sociology entries in Knowledge (XXG) by making it easier for sociologists to become involved in writing and editing them, and second, to facilitate professors giving Knowledge (XXG)-writing assignments
245:
We'd all like to see Knowledge (XXG)'s coverage improve in the areas where it's weakest, but we're all volunteers. Not everyone focuses on content; and those who do generally have areas of expertise. With standards for sourcing higher than they used to be, it's no longer enough just to be interested;
1406:
I think that's a good characterization of what articles are likely to get Randyized. That sounds like something that should be included in professor orientation too, so that the professors understand some of the nuances of article selection for their students. It's not just Randy-magnets that need
1025:
that "the biggest return on investment we can get here is the academics, not the students"; the students of today are the academics of tomorrow, and any work we can to do make them feel positive about Knowledge (XXG) is an investment in the future; academia in 20 years time will certainly use online
975:
There is another way of getting more academics involved. By suggesting that they write articles/papers on the subject of their choice, get it peer reviewed by some colleagues, and put it up on a website, preferably with a CC by (something) license. Then, they can post a link in the relevant article,
786:
for some reason people who spend 40 years learning everything they can about, say, the Peloponnesian War — and indeed, advancing the body of human knowledge — get all pissy when their contributions are edited away by Randy in Boise who heard somewhere that sword-wielding skeletons were involved. And
440:
If we manage the influx of academic interest correctly, Knowledge (XXG) will acquire an institutional connection to academia that will be a source of new content for our articles and an intellectual resource to assist with long-term growth. Knowledge (XXG) does not need to beg for respectability any
339:
Right now, the US program is focused on scaling: recruiting large numbers of classes, increasing ambassador recruitment to support those classes, and training campus support staff to teach students and instructors as much as possible about editing so that they don't cause more harm than good. I want
330:
One of the goals of the USEP is to recruit new editors, but I would be very surprised if it's turning out to be a cost-effective recruitment method. The students edit because they have to, so the retention rate is abysmal. We do have some successes at turning students into editors, but they are rare
1484:
I think there are two main groups in academia that are apt to be fruitful for contributions to WP in the future — students (who are already here) and emeritus professors. The time aspect is major, with publish-or-perish more than just a catchy slogan and teaching duties sapping all available hours.
1450:
I have been encouraging my colleagues in academia and education publicly to criticize Knowledge (XXG) at every turn because students use it too frequently and there is no rigorous content review process. I also advocate that if academics feel the need to correct or add something to Knowledge (XXG),
1182:
Mike, thanks for your thoughtful essay on Knowledge (XXG) and academia. I did want to correct one misconception here as the person running the WMF's U.S. Education Program. We absolutely agree that the USEP should shift its concern from quantity to quality. Since I started, I've been working on how
1105:
I also agree this is a good article and important subject. However, as both a professor and Wikipedian, I must confess I am mildly skeptical about the participation of my fellow academics in Knowledge (XXG), especially those who aren't particularly interested in becoming Wikipedians in the cultural
1055:
This is a good article - thanks for writing it Mike. However, I disagree with your view that "all our resources" should be directed away from students and towards academics. While I agree that class-based editing is generally not worth the effort (and when it goes wrong can cause a lot of upset and
936:
These days, though, I don't think there is the same problem with academics having references demanded, they see how extensive the referencing on Knowledge (XXG) tries to be, and are happy to provide references. And often there are "semi-experts" or "lay experts" who will help with that. There is a
1336:
While reviewing the psychology student's edits (see below), the same mistakes kept coming up. And every now and then, WP:MED see some new "expert" writer appear who just doesn't get WP. Such as writing an article like a review of the primary literature. Or just not understanding WP:V and WP:NOR at
1030:
is the opportunity to lay good foundations regarding Knowledge (XXG). However, I do think that Wiki-in-Edu is best driven from the top down; if the lecturers and the other 'wikipedia trainers' (online Ambs, etc) have a solid knowledge of fundamental Knowledge (XXG) ethics, and indeed an enthusiasm
960:
Obviously recruiting experts is important, but academic experts make their names by disagreeing with one another, and I doubt Knowledge (XXG) is geared up to deal with that. I think Mike has missed one important point though, which is that if properly monitored students can learn the importance of
363:
The USEP, and any other similar programs, need to change their focus. Instead of trying to scale the program to the largest possible number of classes, we should be focusing all our resources on working with academics who are receptive to the idea of becoming editors. We can provide a good deal of
266:
isn't going to get a publication credit that will help his or her career. And Wikipedians don't respond well to a display of credentials; an editor who says "I'm a professor and I know this is right" is going to be met with "show me the citation", which they may find hostile, particularly when the
1628:
The numbers are very much what I'd have expected in the first place for this type of an assignment. I don't see what's so negative - unless somebody is under the illusion that students are going to be jumping through "learn how to be a good Knowledge (XXG) editor" hoops for 3% of credit. In fact,
1301:
Lastly, I don't believe the Randy in Boise issue is as bad as some make it. Sure it happens, but some of our burnt-out experts sought conflict by playing whack-a-mole with the POV pushers. There are thousands of serious and important topics on WP that one can edit without conflict, where the edit
1297:
I note the comment above about the difficulties getting the professors to edit, and some of the issues academics face adapting to the goals of WP. But unless these students can be inspired to write for WP rather than just instructed to write for WP, we won't retain them. Unless they can get ready
1161:
wary of these initiatives because I worry they sometimes take on a tone of: let's figure out how to fix Knowledge (XXG) culture so it's more like academic culture, when I think Knowledge (XXG) culture has the better side of the argument in some (though not all) cases. :-) But it doesn't seem like
857:
I'm rather a fan of this piece. One of the biggest problems with the WMF is that they're apparently all dreamers. They charge ahead pushing noble ideas without adequately assessing the impact or laying out infrastructure ahead of time. Whatever else is accomplished by these noble charges, a great
1291:
Thanks for this essay Mike. I agree with a lot of it. Successful article writing depends on editor ability, motivation and support. Students studying the elementary aspects of their subject are no more able to write or contribute to a professional encyclopaedia article than any reasonably bright
1241:
I agree that it will be hard to get the professors to edit. I don't think it could be a fellowship project, unless I'm misunderstanding the parameters for that; fellowships aren't for things that could/should be done by volunteers, so I don't think it would be possible to make this a fellowship
1016:
I half-agree - or maybe a bit more than half - with "USEP should change its focus in two ways: from growth to quality, and from students to academics". For the first sub-point, I totally agree; I made the analogy in Boston, when Sue demonstrated the contributions of the PPP with a large stack of
1463:
essays by contributing as IPs — identifying themselves with their real name and academic position in their talkpage signatures if they think it important. Even though there are parts of the encyclopedia that are locked out to IP editors, I've found that contributing as an IP removes much of the
1379:
The Randy problem is specifically bad in areas affected by off-wiki ideological conflicts. For instance, I can't imagine any academic wishing to waste time editing our articles on climate change or the Israeli-Palestinian conflict when there are legions of Randies ready to jump in and trash the
1222:
followup (linked in Doc James' comments at the end of this page) is another way to do it. These are labour intensive, unfortunately, but if the students are actually generating more work for the community than they're adding in value we need to know that (and conversely it's good to know which
928:
It behooves us though not to forget the flip side of this equation - academics are subject to the same foibles as other Wikipedians and people in general. When things get a little deeper there are turf wars in many academic fields, discussions, sometimes friendly, sometimes bitter, run in the
433:
Students will disappear after a single year; academics, if we help them, will return year after year. Any academic can run a class using Knowledge (XXG), with or without formal involvement with the USEP, but the USEP should focus its resources on instructors who show willingness to engage with
411:
The quality metrics which were put in place for the Public Policy Initiative need to be restarted for the USEP and other programs. The USEP will have a cost to the community in terms of support, and we need to understand the benefit to article quality to be sure that it's worthwhile. Currently
1078:
academics. My argument is that if we give concentrated high-quality support to those academics who are willing to learn about Knowledge (XXG), we will be expanding the pool of support for students, which in turn will enable us to support more classes since those academics will need less from
1217:
Hi, Jami -- thanks for the comments. I'm glad to hear that we're focusing on quality; that's great news. That prompts two questions: first, do you have a goal for the right number of students per ambassador? At the moment I think we're running at something like 75 or 100 students for each
1188:
One of the ways we are trying to prevent some of these problems is by altering our training materials. After some great feedback on the USEP talk page, we are creating a professor orientation that will expose them upfront to some of the challenges students are facing in adapting traditional
530:
For topics with a large literature, where most of the statements synthesize dozens if not hundreds of sources ... it is simply futile to demand that readers with no subject matter knowledge be able to verify articles on large topics – only an expert reviewer can properly do it. I seek an
922:
We do desperately need experts, not so much because of specific deep knowledge, but because they understand how the balance of items at what is a basic level in their field should go - what is due weight, what is fringe but interesting, what is interesting but irrelevant, what has been
1194:
I'm also intrigued by the idea of focusing on professors editing rather than students, but as others have mentioned here, getting that to work has been difficult. But I'd love to see someone take this on and develop a pilot program to get professors editing -- perhaps this could be a
434:
editors on Knowledge (XXG); who participate in reviewing and managing their own students' contributions; and most of all who show interest in working on articles themselves. In return we can supply ambassadors as teaching assistants – mentors, in Knowledge (XXG) terminology.
276:
to students in their courses”. This is a promising start, but it would be sad if the experts who might arrive as a result of these initiatives are driven away, as many new editors are driven away, by the difficulty of learning how to engage with Knowledge (XXG).
160:
Mike Christie has been editing Knowledge (XXG) since he was stuck in a hotel in South Korea with jetlag in early 2006. He works primarily on featured articles, and has been involved for about a year with WMF initiatives related to education. He attended the
590:
793:
bringing in lots of academics. Otherwise academics who join the project will leave in frustration, and word will get around that contributing to Knowledge (XXG) is nothing but an exercise in futility. If that happens we'll be worse off than before.
391:
The community needs to engage with the Global Education Program, and particularly with the USEP, the largest of the country EPs. These programs are largely driven off-wiki, since they are WMF operations rather than community initiatives. Two key
503:
1017:
photocopying paper - without wishing to disparage the tremendous work done, I felt that the whole idea of merely counting added-text was wrong-wrong-wrong, and we may as well have demonstrated it with a palette-full of toilet paper; we
1501:
I cannot express how strongly I agree with the sentiments here. I am saying this as somebody who has been involved with the teaching with Knowledge (XXG) approach (and with teaching students) for quite a few years. Hear, hear!
1110:
excellent in that regard). But as for more specific problems. One, I've found academics to be extremely bad at restraining themselves from self-promotion. More edits by professors than you'd think are to things as absurd as
340:
to be very clear that "more harm than good" is not a theoretical problem; the Indian EP (which has the potential to be an even bigger asset to the encyclopedia than the USEP) required a great deal of cleanup work, as
1298:
access to been-there-done-that support from their institution, they will struggle. There is a risk that a poor experience of writing on WP as a student could put a generation of young editors off the idea altogether.
441:
more; it is already widely used by academics as a starting point for research, and sometimes for more than that. We need to accept our respectability, and plan to learn from – and teach – the academic community.
762:
731:
721:
586:
423:
323:
signed up as part of the USEP – so many, in fact, that it's straining the ability of the limited number of volunteer ambassadors to effectively assist the students. In addition to the USEP, there is a
316:
736:
1189:
assignments to a Knowledge (XXG) assignment. Another aim of this orientation is to encourage professors to make edits themselves, as it will give them a better foundation in Knowledge (XXG)-editing.
726:
271:(APS), for example, recently launched an initiative to ensure, among other things, that Knowledge (XXG)'s "articles about psychological research and theory are accurate, up-to-date complete". The
1056:
wasted effort for the poor students and their lecturer), given that a high proportion of our most active editors are university students, I suspect it's an OK way to bolster their numbers. Surely
427:
320:
711:
341:
76:
716:
699:
1074:
We should support the students the best way we can, but right now the USEP is trying to expand as quickly as it can, and I don't think that's going to lead to good support for students
1658:
1232:
I also am glad to hear about the expanded training materials and professor orientation. I assume this will be done by the campus ambassadors? I think this is a very positive step.
179:
693:
52:
41:
416:; these aren't inherently harmful but anyone who has spent time reverting poor student contributions this semester will appreciate that a quality metric is at least as important.
787:
they get downright irate when asked politely to engage in discourse with Randy until the sword-skeleton theory can be incorporated into the article without passing judgment.
422:
Growth without support will lead to stress on the community and poorer quality. Instead we should work out what it means to support a class properly – there are far too few
207:
Knowledge (XXG) has come a long way by relying on self-organized volunteer labour. It's hard for most current editors to imagine what it was like years ago, when pages like
1349:
Sounds like a good idea. I left a note on Jami Mathewson's talk page about this suggesting she contact you; I would assume she's coordinating the orientation material.
603:
1546:
Doc James recently left this note on my talk page, which I'm reproducing here: the link show exactly what value was introduced by each student edit for a given class.
1711:
327:
for school programs that did not form via the USEP that lists another 55 classes, and there are also active classes that do not declare themselves at either page.
929:
correspondence pages of learned journals for months, friends cite friends, peer review is not as "pure" as we would like, this is without systemic problems like
397:
372:
21:
1686:
324:
315:(the US branch of the Global Education Program) now recruits academics in all areas of study, and matches them with Wikipedians who are willing to act as
1681:
1676:
226:; there are specialists – some professionals in their field and some who are pursuing a hobby interest – who focus on highly detailed areas, such as
1314:
I agree; and I think that's something that could be included in the additional orientation Jami Mathewson mentions above. Do you want to write it?
1380:
articles (and their authors) whenever the latest faux outrage hits Fox News or Rush Limbaugh. For a more micro example of the problem, I can offer
1671:
799:
667:
218:
One result of this change is that you can get a sense of what interests Wikipedians by looking at which areas have good coverage. It's not just
268:
1123:
one you personally are strongly invested in, and try to cite as sources papers written by people who are not you. Medium-low luck so far. --
633:
1600:
1424:
1411:. The teacher for that class either did not understand or did not successfully convey to the students what makes an article encyclopedic.
1362:
1327:
1255:
1151:
1092:
909:
430:; the ratio currently seems to be about one ambassador to two classes, which means many students will never hear from an ambassador at all.
319:, to teach and guide the students and instructors. There has been a good deal of interest in the program, and the fall 2011 semester saw
1596:
1469:
1420:
1358:
1323:
1251:
1147:
1088:
977:
905:
287:), an early pioneer of encouraging engagement between Knowledge (XXG) and higher education who documented his experiences in the essay "
1629:
that about a quarter of them tried at least a little, is quite positive to me, as I'd expect a much smaller amount to try it at all. --
795:
670:
the use of text plucked from a Knowledge (XXG) article in a symposium paper by the president of the American Sociological Association.
1666:
681:
272:
242:
particularly interest Wikipedians by looking to see what areas are poorly covered: psychology, for example, or major topics in art.
46:
32:
17:
262:
How can we attract experts in these topics to edit Knowledge (XXG)? A professor of physics who spends time improving an article on
227:
367:
1119:
area, so e.g. if you're a theoretical computer scientist, start with an article on an important theoretical-CS result that is
1106:
sense, i.e. understand the new project they're participating in and its values (some of the mathematics-professor Wikipedians
1619:
165:
in Boston in July 2011. He would like to thank all the editors who gave helpful feedback on earlier versions of this article.
1162:
that's what's happening in this case, so I agree more outreach to find the professors who would be good editors is good. --
483:
858:
number of completely avoidable problems could have been... well... avoided, if the WMF were only able to start small and
1573:
937:
general move too far in demanding references, I would agree, mostly due to an out of context quote being bandied around.
649:
The PPI quality metric was quite labour-intensive, which may make it more difficult to implement. The Foundation does
292:
376:
1641:
1623:
1604:
1581:
1535:
1514:
1494:
1477:
1445:
1428:
1401:
1366:
1344:
1331:
1309:
1283:
1259:
1208:
1171:
1132:
1096:
1069:
1048:
1007:
985:
968:
950:
913:
886:
845:
825:
803:
1204:
965:
933:. We recently had fallacious figures from an academic reprinted in Signpost, so this is not just a hypothetical.
616:
200:
636:
to the APS initiative for an example of an instructor's response. The drive to expand the USEP was visible at the
944:
839:
543:
476:... most of the psych articles we could consider "important" on any scale in the psych realm are utterly dismal."
569:
1692:
1577:
199:
campus ambassadors at Georgetown University, Washington D.C. posing with a repurposed quotation of philosopher
1592:
1473:
1416:
1354:
1319:
1247:
1143:
1084:
981:
901:
877:
393:
1558:
413:
404:, the Global Education Program Manager. I suspect the best place to discuss these programs with the WMF is
1557:. Things do not look that upbeat in Canada. Have been involved with a great deal of outreach as seen here
1555:
1408:
1060:
resources should continue to be directed towards peer to peer recruitment and support of student editors?
896:
small successes followed by gradual scaling, so perhaps the issue here is that the scaling is too rapid.
405:
1272:
Leveraging knowledge by teaching those who want to teach is a powerful way to accelerate organic growth.
1200:
962:
289:
Was introducing Knowledge (XXG) to the classroom an act of madness leading only to mayhem if not murder?
288:
1436:
should be aware that this may have a downside in new editor retention and the like. Anyway, good job.
1196:
998:
journal a paper is published in is important, and the journals end up with the copyright on the paper.
467:
192:
419:
The USEP should change its focus in two ways: from growth to quality, and from students to academics.
941:
836:
813:
380:
300:
178:. The Signpost welcomes proposals for op-eds. If you have one in mind, please leave a message at the
1397:
1279:
1460:
1615:
1588:
1412:
1350:
1315:
1243:
1167:
1139:
1128:
1080:
897:
863:
821:
360:, and as far as I can tell the WMF regards growth as a high-priority goal for all such programs.
153:
1456:
1452:
785:
457:
I'm talking about content here – it's easy to find articles that need copyediting or formatting.
93:
1635:
1508:
1490:
1003:
758:
251:
231:
123:
312:
196:
1569:
1065:
930:
808:
I don't think that's actually the problem you're making it out to be. The expert input into
663:
556:
375:
speaking on the topic of "Knowledge (XXG) in higher education: a new model for teaching" at
170:
The views expressed are those of the author only, and do not necessarily represent those of
1551:
1219:
892:
357:
353:
349:
308:
103:
1441:
1342:
1307:
401:
299:
In addition to proposals such as the APS's initiative, there is an ongoing project by the
235:
223:
219:
133:
528:
For example, a Wikipedian who is a professional neuroscientist recently commented that "
1393:
1381:
1275:
1041:
637:
162:
1705:
1611:
1531:
1163:
1124:
817:
582:
284:
502:). For the psychology articles you can get an idea of the assessed quality from the
1631:
1504:
1486:
999:
620:
495:
263:
247:
113:
279:
143:
1561:
1407:
to be avoided; just today I saw someone having to userfy student essays such as
1385:
1061:
831:
809:
478:
and "in my area of the visual arts ... most articles on major topics are crap (
1437:
1384:, an article I've been working on for a while. It's worth contrasting that to
1338:
1303:
516:
479:
1523:
1034:
994:
Not really practical under the current system of academic funding/prestige.
208:
304:
1274:
The hard part is stimulating interest in teachers to really take it on. ~
1527:
834:, this was a carefully engineered collaboration, so may not be typical.
1451:
they can most easily avoid the special sorts of pain alluded to in the
1390:
because nobody has any ideological difficulties with the subject matter
487:
1464:
problematic aspects of Knowledge (XXG) culture and allows content to
585:
of the PPI for details; you may also be interested in looking at the
491:
174:
or its staff. Responses and critical commentary are invited in the
246:
you also have to have access to good sources. For topics such as
602:
See, for example, some extended discussion on this topic at the
51:
680:
31:
1526:
body of knowledge is the much better objective. Cheers.
1021:
get away from that extended version of "editcountitis". I
283:
Academic and Knowledge (XXG) editor Jon Beasley-Murray (
774:
767:
747:
650:
529:
499:
471:
412:
metrics for quantity exist, such as pages that measure
212:
303:
to get academics to engage with Knowledge (XXG) – the
891:
I don't think this is entirely fair to the WMF; the
772:If your comment has not appeared here, you can try
383:held at the University of Göttingen in June 2011
619:at the strategy wiki; that page is linked from
472:"the quality across the board in psychology is
860:stay small until all the kinks were worked out
400:, the Global Education Program Director, and
307:(GEP). This grew out of a project called the
187:
8:
1550:We are currently crunching some numbers at
387:How do we do this? Here's my prescription.
293:Opening up the academy with Knowledge (XXG)
638:Knowledge (XXG) in Higher Education Summit
163:Knowledge (XXG) in Higher Education Summit
1712:Knowledge (XXG) Signpost archives 2011-12
486:, in fact anything to do with sculpture,
468:the Featured Article Candidates talk page
519:for more background on editor retention.
366:
278:
191:
18:Knowledge (XXG):Knowledge (XXG) Signpost
1026:resources in a very different way, and
775:
751:
450:
188:What we're good at – and what we're not
67:
1587:So far it doesn't look very positive.
1560:and the effect has been mixed at best.
1632:Piotr Konieczny aka Prokonsul Piotrus
1505:Piotr Konieczny aka Prokonsul Piotrus
653:that quantity metrics are inadequate.
426:to properly review and help with the
269:Association for Psychological Science
7:
557:the November 2011 issue of Footnotes
335:The future of the education programs
236:early English ecclesiastical history
238:. You can also get an idea of what
371:Global Education Program Director
295:" with fellow academic Wikipedians
53:
28:
784:The elephant in the room is that
757:These comments are automatically
617:the "increase participation" goal
504:Psychology articles quality chart
273:American Sociological Association
68:Knowledge (XXG) in Academe – and
138:
128:
118:
108:
98:
88:
1542:Stats on value of student edits
377:Knowledge (XXG) meets Antiquity
258:Knowledge (XXG) needs academics
197:United States Education Program
789:We need to solve that problem
768:add the page to your watchlist
291:", and co-authored the paper "
1:
1642:16:51, 21 December 2011 (UTC)
1624:20:36, 14 December 2011 (UTC)
1605:11:56, 14 December 2011 (UTC)
1582:19:50, 13 December 2011 (UTC)
1515:16:47, 21 December 2011 (UTC)
1495:03:03, 21 December 2011 (UTC)
1478:17:03, 15 December 2011 (UTC)
1446:00:09, 15 December 2011 (UTC)
1429:15:47, 15 December 2011 (UTC)
1402:21:19, 14 December 2011 (UTC)
1367:17:20, 16 December 2011 (UTC)
1345:20:35, 15 December 2011 (UTC)
1332:15:42, 15 December 2011 (UTC)
1310:20:39, 14 December 2011 (UTC)
1284:19:27, 14 December 2011 (UTC)
1260:11:31, 15 December 2011 (UTC)
1209:19:16, 14 December 2011 (UTC)
1172:13:01, 15 December 2011 (UTC)
1133:13:23, 14 December 2011 (UTC)
1097:11:32, 14 December 2011 (UTC)
1070:10:29, 14 December 2011 (UTC)
1049:03:02, 14 December 2011 (UTC)
1008:21:35, 14 December 2011 (UTC)
986:00:08, 14 December 2011 (UTC)
969:21:03, 13 December 2011 (UTC)
951:20:44, 13 December 2011 (UTC)
914:20:02, 13 December 2011 (UTC)
887:19:42, 13 December 2011 (UTC)
846:20:46, 13 December 2011 (UTC)
826:19:03, 13 December 2011 (UTC)
804:18:12, 13 December 2011 (UTC)
796:Short Brigade Harvester Boris
640:, held this summer in Boston.
621:one of the USEP project pages
484:Italian Renaissance sculpture
175:
466:Recent quotes by editors at
1536:22:49, 7 January 2012 (UTC)
1409:Global Media and Newspapers
1157:That seems reasonable. I'm
1113:an article about themselves
1728:
1197:meta:Wikimedia_Fellowships
544:Knowledge (XXG) Initiative
414:student editing statistics
201:Pierre Teilhard de Chardin
424:active online ambassadors
354:recent changes patrollers
583:the statistical analysis
309:Public Policy Initiative
305:Global Education Program
591:example assessment page
381:Knowledge (XXG) Academy
1610:Ouch! Indeed not! --
765:. To follow comments,
685:
570:the wrap-up blog entry
384:
296:
204:
36:
684:
370:
282:
195:
35:
814:Human papillomavirus
761:from this article's
301:Wikimedia Foundation
517:Editor Trends Study
350:new page patrollers
752:Discuss this story
732:Arbitration report
722:WikiProject report
686:
662:Steve Kolowich at
428:current class load
406:the USEP talk page
385:
297:
211:consisted of just
205:
42:← Back to Contents
37:
1047:
954:
884:
849:
776:purging the cache
737:Technology report
587:assessment rubric
398:Frank Schulenburg
394:WMF staff members
373:Frank Schulenburg
342:reported recently
252:History of Sweden
47:View Latest Issue
1719:
1695:
1638:
1566:
1511:
1466:speak for itself
1201:JMathewson (WMF)
1046:
1044:
1038:
1032:
949:
931:publication bias
883:
878:
875:
844:
779:
777:
771:
750:
727:Featured content
704:
696:
694:12 December 2011
689:
671:
664:Inside Higher Ed
660:
654:
647:
641:
630:
624:
613:
607:
600:
594:
579:
573:
566:
560:
553:
547:
540:
534:
526:
520:
513:
507:
464:
458:
455:
358:Canadian program
224:military history
176:comments section
156:
142:
141:
132:
131:
122:
121:
112:
111:
102:
101:
92:
91:
59:
57:
55:
54:12 December 2011
1727:
1726:
1722:
1721:
1720:
1718:
1717:
1716:
1702:
1701:
1700:
1699:
1698:
1697:
1696:
1691:
1689:
1684:
1679:
1674:
1669:
1662:
1650:
1649:
1640:
1636:
1562:
1544:
1513:
1509:
1042:
1036:
1033:
879:
872:
868:
864:
781:
773:
766:
755:
754:
748:+ Add a comment
746:
742:
741:
740:
697:
692:
690:
687:
675:
674:
661:
657:
648:
644:
631:
627:
614:
610:
601:
597:
580:
576:
567:
563:
554:
550:
541:
537:
527:
523:
514:
510:
465:
461:
456:
452:
447:
337:
260:
190:
157:
151:
150:
149:
148:
139:
129:
119:
109:
99:
89:
83:
80:
66:
62:
60:
50:
49:
44:
38:
26:
25:
24:
12:
11:
5:
1725:
1723:
1715:
1714:
1704:
1703:
1690:
1685:
1680:
1675:
1670:
1665:
1664:
1663:
1652:
1651:
1648:
1647:
1646:
1645:
1644:
1630:
1585:
1584:
1543:
1540:
1539:
1538:
1518:
1517:
1503:
1498:
1497:
1481:
1480:
1448:
1433:
1432:
1431:
1382:Cyrus Cylinder
1376:
1375:
1374:
1373:
1372:
1371:
1370:
1369:
1299:
1294:
1293:
1288:
1287:
1265:
1264:
1263:
1262:
1236:
1235:
1234:
1233:
1227:
1226:
1225:
1224:
1212:
1211:
1191:
1190:
1185:
1184:
1179:
1178:
1177:
1176:
1175:
1174:
1102:
1101:
1100:
1099:
1052:
1051:
1013:
1012:
1011:
1010:
989:
988:
972:
971:
957:
956:
955:
938:
934:
925:
924:
919:
918:
917:
916:
870:
866:
855:
854:
853:
852:
851:
850:
756:
753:
745:
744:
743:
739:
734:
729:
724:
719:
714:
712:News and notes
709:
703:
691:
679:
678:
677:
676:
673:
672:
655:
642:
625:
608:
604:USEP talk page
595:
574:
561:
548:
542:See the APS's
535:
521:
508:
459:
449:
448:
446:
443:
438:
437:
436:
435:
431:
417:
409:
336:
333:
259:
256:
189:
186:
185:
147:
146:
136:
126:
116:
106:
96:
85:
84:
81:
75:
74:
73:
72:
64:
63:
61:
58:
45:
40:
39:
30:
29:
27:
15:
14:
13:
10:
9:
6:
4:
3:
2:
1724:
1713:
1710:
1709:
1707:
1694:
1688:
1683:
1678:
1673:
1668:
1660:
1656:
1643:
1639:
1633:
1627:
1626:
1625:
1621:
1617:
1613:
1609:
1608:
1607:
1606:
1602:
1598:
1594:
1590:
1589:Mike Christie
1583:
1579:
1575:
1571:
1567:
1565:
1559:
1556:
1553:
1549:
1548:
1547:
1541:
1537:
1533:
1529:
1525:
1520:
1519:
1516:
1512:
1506:
1500:
1499:
1496:
1492:
1488:
1483:
1482:
1479:
1475:
1471:
1470:128.59.169.46
1467:
1462:
1458:
1454:
1449:
1447:
1443:
1439:
1434:
1430:
1426:
1422:
1418:
1414:
1413:Mike Christie
1410:
1405:
1404:
1403:
1399:
1395:
1391:
1387:
1383:
1378:
1377:
1368:
1364:
1360:
1356:
1352:
1351:Mike Christie
1348:
1347:
1346:
1343:
1340:
1335:
1334:
1333:
1329:
1325:
1321:
1317:
1316:Mike Christie
1313:
1312:
1311:
1308:
1305:
1300:
1296:
1295:
1290:
1289:
1286:
1285:
1281:
1277:
1273:
1267:
1266:
1261:
1257:
1253:
1249:
1245:
1244:Mike Christie
1240:
1239:
1238:
1237:
1231:
1230:
1229:
1228:
1221:
1216:
1215:
1214:
1213:
1210:
1206:
1202:
1198:
1193:
1192:
1187:
1186:
1181:
1180:
1173:
1169:
1165:
1160:
1156:
1155:
1153:
1149:
1145:
1141:
1140:Mike Christie
1136:
1135:
1134:
1130:
1126:
1122:
1118:
1114:
1109:
1104:
1103:
1098:
1094:
1090:
1086:
1082:
1081:Mike Christie
1079:Wikipedians.
1077:
1073:
1072:
1071:
1067:
1063:
1059:
1054:
1053:
1050:
1045:
1040:
1039:
1029:
1024:
1020:
1015:
1014:
1009:
1005:
1001:
997:
993:
992:
991:
990:
987:
983:
979:
978:117.198.51.14
974:
973:
970:
967:
964:
959:
958:
952:
947:
946:
943:
939:
935:
932:
927:
926:
921:
920:
915:
911:
907:
903:
899:
898:Mike Christie
894:
890:
889:
888:
885:
882:
876:
874:
861:
856:
847:
842:
841:
838:
833:
829:
828:
827:
823:
819:
815:
811:
807:
806:
805:
801:
797:
792:
788:
783:
782:
778:
769:
764:
760:
749:
738:
735:
733:
730:
728:
725:
723:
720:
718:
715:
713:
710:
708:
707:Opinion essay
705:
701:
695:
688:In this issue
683:
669:
665:
659:
656:
652:
646:
643:
639:
635:
634:this response
629:
626:
622:
618:
612:
609:
605:
599:
596:
592:
588:
584:
578:
575:
571:
565:
562:
558:
552:
549:
545:
539:
536:
532:
525:
522:
518:
512:
509:
505:
501:
497:
493:
489:
485:
481:
477:
475:
469:
463:
460:
454:
451:
444:
442:
432:
429:
425:
421:
420:
418:
415:
410:
407:
403:
399:
395:
390:
389:
388:
382:
378:
374:
369:
365:
361:
359:
355:
351:
347:
343:
334:
332:
328:
326:
322:
318:
314:
310:
306:
302:
294:
290:
286:
281:
277:
274:
270:
265:
257:
255:
253:
249:
243:
241:
237:
233:
229:
225:
221:
216:
214:
213:two sentences
210:
202:
198:
194:
184:
183:
181:
177:
171:
167:
166:
164:
155:
154:Mike Christie
145:
137:
135:
127:
125:
117:
115:
107:
105:
97:
95:
87:
86:
78:
71:
65:Opinion essay
56:
48:
43:
34:
23:
19:
1654:
1586:
1563:
1545:
1465:
1389:
1271:
1270:
1158:
1120:
1116:
1112:
1107:
1075:
1057:
1035:
1027:
1022:
1018:
995:
940:
880:
865:
859:
835:
790:
706:
700:all comments
658:
645:
628:
611:
598:
577:
572:for details.
564:
551:
538:
524:
511:
496:Romantic art
473:
462:
453:
439:
386:
362:
345:
338:
329:
298:
264:weak isospin
261:
248:Architecture
244:
239:
217:
206:
180:opinion desk
173:
172:The Signpost
169:
168:
159:
158:
69:
1693:Suggestions
1386:Hoxne Hoard
832:Hoxne hoard
810:Hoxne hoard
759:transcluded
717:In the news
668:remarked on
651:acknowledge
498:etc etc)" (
325:signup page
317:ambassadors
220:video games
1657:. You can
1653:It's your
1637:talk to me
1510:talk to me
945:Farmbrough
840:Farmbrough
480:Indian art
321:77 classes
82:Share this
77:Contribute
70:vice versa
22:2011-12-12
1687:Subscribe
1564:Doc James
1461:WP:MMORPG
1394:Prioryman
1276:Ningauble
1199:project?
1028:right now
923:debunked.
763:talk page
666:recently
402:Annie Lin
209:Neurology
1706:Category
1682:Newsroom
1677:Archives
1655:Signpost
1620:contribs
1612:jbmurray
1597:contribs
1574:contribs
1457:WP:CRUSH
1453:WP:RANDY
1421:contribs
1359:contribs
1324:contribs
1252:contribs
1164:Delirium
1159:somewhat
1148:contribs
1125:Delirium
1117:specific
1089:contribs
1023:disagree
966:Fatuorum
906:contribs
818:The Land
515:See the
474:horrific
470:include
346:Signpost
285:Jbmurray
124:LinkedIn
104:Facebook
20: |
1659:help us
1601:library
1524:emeging
1487:Carrite
1425:library
1363:library
1328:library
1256:library
1152:library
1093:library
1000:Gerardw
963:Malleus
910:library
873:anguard
488:Baroque
344:in the
240:doesn't
228:Banksia
114:Twitter
1552:WP:MED
1459:, and
1220:WP:MED
1062:Nick-D
791:before
589:or an
492:Rococo
232:lemurs
134:Reddit
94:E-mail
1672:About
1578:email
1554:here
1438:Cla68
1339:Colin
1304:Colin
1037:Chzz
996:Which
546:page.
445:Notes
250:, or
234:, or
16:<
1667:Home
1616:talk
1593:talk
1570:talk
1532:talk
1491:talk
1474:talk
1442:talk
1417:talk
1398:talk
1355:talk
1320:talk
1280:talk
1248:talk
1205:talk
1168:talk
1144:talk
1129:talk
1085:talk
1066:talk
1058:some
1019:must
1004:talk
982:talk
942:Rich
902:talk
881:Wha?
869:ven
837:Rich
822:talk
800:talk
632:See
615:See
581:See
568:See
555:See
500:diff
396:are
379:, a
313:USEP
222:and
144:Digg
1599:-
1528:N2e
1423:-
1361:-
1326:-
1254:-
1150:-
1121:not
1108:are
1091:-
908:-
893:PPI
830:On
812:or
152:By
79:—
1708::
1622:)
1618:•
1603:)
1595:-
1580:)
1576:·
1572:·
1534:)
1502:--
1493:)
1476:)
1455:,
1444:)
1427:)
1419:-
1400:)
1365:)
1357:-
1330:)
1322:-
1282:)
1258:)
1250:-
1207:)
1170:)
1154:)
1146:-
1131:)
1095:)
1087:-
1076:or
1068:)
1043:►
1006:)
984:)
948:,
912:)
904:-
843:,
824:)
802:)
494:,
490:,
482:,
352:,
230:,
1661:.
1634:|
1614:(
1591:(
1568:(
1530:(
1507:|
1489:(
1472:(
1440:(
1415:(
1396:(
1353:(
1341:°
1318:(
1306:°
1278:(
1246:(
1203:(
1166:(
1142:(
1127:(
1083:(
1064:(
1002:(
980:(
953:.
900:(
871:M
867:S
848:.
820:(
798:(
780:.
770:.
702:)
698:(
623:.
606:.
593:.
559:.
533:"
506:.
408:.
203:.
182:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.
