evaluation of united learning I ... topic 2

UNIT 1 ... ..THEM # 2

1.- On the design and administration of objective tests F.L.T.A

The test administrator has the mission and responsibility that the test objectives are successful. This role involves quality and a promotion of tests, administration of resources and solution of problems that prevent the achievement of the tests. He is responsible for the creation of the Summary of Evaluation Tests and Test Plan.

In this sense, the Test Manager is an important key to achieve that the administration of the test is carried out with the expected criteria and that are established in this document. Consequently, those who play this role should be noted for presenting a high degree of responsibility, honesty and impartiality; in addition to maintaining a friendly and cordial treatment with the supporters.

Multiple choice or objective tests consist of a set of clear and precise questions that require a brief response from the student, generally limited to the choice of an option already provided. The objective term refers to the conditions of application of the test as well as to the treatment and subsequent analysis of the results, but this does not imply a greater objectivity in the evaluation of the student's performance. Some aspects of an objective test are:

 • The selection of contents

• The writing of the questions or items

• Correction and punctuation

• Presentation of the same.

Selection of contents of the test: In the selection of the contents, all those that are relevant in relation to the domain to be measured must be specified in an operative way and referred to the objectives sought as well as according to the abilities of the students. A table of specifications must be drawn up consisting of a double-entry table that indicates the contents and competences (skills that are put into play when considering new situations) in the corresponding columns and rows. At each intersection, a measurable objective is presented, represented by the items proposed in each case, whose number is specified in the table. This must be compared with the table of results of the test once it has been corrected. It is observed that the subjects "material systems" and "scientific procedure" correspond to 11 and 8 items respectively, while the remaining ones correspond to 5 and 6. Regarding the competences to be evaluated in the students (rows), the The test is aimed at evaluating 3 types of skills: basic level (identification, recognition and reproduction of information), intermediate level (interpretation of information) and higher level (production of information). Approximately one third of the test is devoted to each type of skill.

The most common norms for the elaboration of a test could be synthesized in the following ones: • In the test the contents to be measured must be considered, which must be explicit and known a priori by the students.

• The questions should refer to the most important aspects of the domain, containing them.

The number of questions or items for each aspect of the domain should be proportional to the importance of the contents to be evaluated.

• You must know, prior to the preparation of the test, what are the objectives achieved as well as the skills or competencies developed that have to be evaluated.

• Questions should be presented in a way that implies increasing difficulty. They are usually placed at the beginning the simplest, then an increase in difficulty to the maximum and finally some issues of less difficulty in small amount.

• It is convenient that the test be submitted to the opinion of experts (other colleagues of the subject, specialists in the subject, pedagogues, etc.)

Writing the test Before considering the form of writing itself, it is important to clarify that it is understood by question. Mateo (2000: 74) states that "usually a subject is understood as a question about which one is going to work." Some characteristics of the items:

a) They must adhere to the competences and contents pre-established in the specifications table.

b) They must be independent of each other and exhaustive in terms of the information necessary for their resolution.

c) The response of an item can not and should not be a condition for the resolution of any of the following items.

d) They should not include fiction names or brands of products used in the mass media.

e) In a test items of varying degrees of difficulty must be considered. The item consists of a base or root expressed in an assertive or interrogative form, which presents the problem situation and a series of response alternatives.

f) Clarity, brevity and intelligibility.

g) Be written in direct and understandable language for the respondent.

h) Refer to a single content.

i) Be expressed, preferably in a positive way.

The distractors arise from the most common errors that students present in their answers, associated with the question posed and are not "just filler" statements, little credible. The correct answer or key 7 must be presented randomly between the alternatives whose ideal number is four to five, never less than three, so that errors of chance can be controlled acceptably. Each item must be analyzed in order to determine its characteristics and facilitate its location in a database.

Criticism of objective evidence

 • It is usually considered that this type of tests assess rote knowledge to the detriment of reasoning.

• The fact of choosing the best option in a response reinforces selective thinking more than the mental processes directed to the construction of knowledge.

 • Chance can be an element of distortion in the evaluation of knowledge.

• The brevity and stereotyping of the tests can facilitate copying between students.

 • The preparation and design is expensive due to its own presentation, its preparation is complex since it requires great dedication of qualified human resources as well as materials.

  Advantages of objective tests

 • It is easy to apply and correct what makes them a choice in contexts of student mass.

 • Has the possibility to cover broad domains of learning.

 • It allows detecting conceptual errors and informing about the quality of complex learning.

 • The correction of the test is less affected by subjective aspects inherent to the corrector. • It enables the identification and quantification of the level of compliance with the objectives proposed to students with the subsequent possibility of rethinking them or reinforcement of unconsolidated topics. • Admits the application of multiple statistical analyzes whose results make possible the various subsequent decision making.

What steps does the design of an objective test involve? • Develops a theoretical framework of reference about how evaluation is conceived (if it is equivalent to measurement, if it is part of the learning process, if it points to understanding, etc.). The content to be evaluated must also be predetermined, eg. If you will evaluate proposed goals, achievements within a certain period, development of certain competences, different types of content taught, among many other possibilities. • Prepare a table of specifications appropriate to the test to be built.

• Make a first approach to the test to be proposed, without alternative answers.

• Apply it to a sample of audience equivalent to the recipient (first piloting) and analyze the problems of the respondents during the performance of the test.

• Analyze the answers obtained, classifying them into: correct, partially correct and incorrect.

• Restructure the test considering the types of response obtained and the processes performed by the respondents.

• Posing the alternatives of possible answers, elaborating the distractors from the mistakes made in the pilot test (see box profile of the item).

 • Set the level of difficulty of the answers

• Request the review by other technicians of the possible activities and alternative answers.

• Organize several forms of preliminary tests using the designed activities.

• Validate the activities proposed in the tests, applying them to a large enough audience and validate them pedagogically from the experts' judgment in the area (second piloting).

• Prepare the final version of the test considering the adjustments that arise from statistics and experts, in case it is going to be applied massively.

• The test score must be previously standardized to know the level of competence of the respondents in a statistically significant way. Before drawing conclusions based on the results of the test, it is necessary to carry out an analysis of the quality of the test as an evaluation instrument.

1. What is the most appropriate number of items?

Initially, the more items include the most reliable and valid test will be the evaluation. Some 40 items can be included in a one-hour test.

The number of items per thematic block should be defined according to the importance assigned in teaching.

 2. To avoid clues that make it easier to answer correctly without knowing the answer:

• Avoid lack of grammatical agreement

• Prevent the correct answer from being longer, more elaborate

• Avoid absurd alternatives

• Avoid the use of very specific determinants (as always, never, etc.)

 3. When in doubt: Question properly said or unfinished phrase

• Questions are slightly easier than if they are incomplete sentences

• The questions themselves raise somewhat the reliability (around .065) and the validity (correlations with other criteria) of the entire test

 4. On the use of negative particles (no, never, except, less) in the formulation of the questions:

As a general rule, negative particles should not be included in the formulation of the item because they lend themselves to mistakes even if they know the answer. (underlined, in italics or in CAPITAL LETTERS)

 5. Regarding the information contained in the items:

It is preferable to keep the questions short and not provide more information than necessary.

 6. On the use of "all of the above" and "none of the above" as possible answers:

o The answer "all the above" is not recommended because you can build an important clue to discard some alternatives.

o Regarding the use of "none of the above" in response:

o Questions to consider for its use:

 7. Regarding the most adequate number of answers, there are several positions based on multiple investigations. Overall, it is considered that:

• Two alternatives (one correct and one incorrect, or the true-false classic) discriminate better in the upper part of the distribution: those who know more are more differentiated, but in the rest of the distribution they are all more undifferentiated

• Three alternatives discriminate and report better in the center of the distribution (the "best" and the "worst" are less differentiated)

• Four or more alternatives give better results in the lower part of the distribution. Those who know less tend to guess more, and have more opportunities to make mistakes

  8. In general, the True-False questions tend to be:

• Easier (guessing more correctly)

• Less discriminant

• The reliability of the whole test is less, keeping the number of items constant

2.-Integral tests or integration tests an integral test is a graphic presentation of the final product that will be delivered and the way in which customers interact with it. As are also those that are made in the field of software development once the unit tests have been approved and what they prove is that all the unit elements that make up the software work together correctly by testing them in a group. It focuses mainly on testing the communication between the components and their communications whether hardware or software.

The integral tests have to be applied just after having carried out each unit test with the intention of testing the methods applied in the development. If there are no code problems and the unit tests have been successfully completed, it will be possible to pass the integral test to make sure that at this point no problem occurs in the combination of unit elements. The main reason is that the integral test carries out the joint review of the different elements that are present in order to form the software. The check is made to see that everything works in a proper way as a whole, since it is not strange that there are alterations in performance. If the communication between the different components present in the software is functional. Communications are also checked invariably whether they are represented with software or with hardware. Communications are also checked invariably whether they are represented with software or with hardware.

During this process in which the different types of integration are verified, the specialists will have to assemble the independent modules, shape the software

One of the advantages of this protocol lies in the opportunity to carry out tests in a parallel manner, which provides extra flexibility in the scheduling process. For this purpose, we will choose the frameworks in which the tests can be combined with the development and with the simplified supervision of the processes, especially in those cases where the tests can be a little more complex. The result will ensure that the software project can advance to its next phase before it is finalized.

In this case we test how is the interaction between two or more units of the software.

This type of tests verify that the components of the application work correctly acting together.

Following the previous case, the integration tests are those that would verify that an email has been sent, the real connection with the database etc.

This type of tests are dependent on the environment in which they are executed. If they fail, it may be because the code is fine, but there is a change in the environment.

For example, we could also use JUnit to perform integration tests, if we do not do mocks or stubs, and we focus on testing the behavior of the components as a whole.

Pragmatic or pragmalinguistic test is a subfield of linguistics, also studied by the philosophy of language, communication and psycholinguistics, which is interested in the way in which context influences the interpretation of meaning. The context should be understood as a situation, since it can include any extralinguistic aspect: communicative situation, knowledge shared by speakers, interpersonal relations, etc. The pragmatic takes into account the extralinguistic factors that condition the use of language, that is, all those factors that are not referenced in a purely formal study.

It can be said that it is also a type of test that is characterized by the use of the example as elements of conviction, using a natural language. The following describes two large groups of demonstrations that are related to the evidence presented by Balacheff (2000). Empirical demonstrations and deductive demonstrations, in the first group a characterization of the examples that students can use in their validation process is made, in this group where the practical tests can be identified, then the two groups are described: Marrades and Gutierrez 2000

Empirical demonstrations: Characterized by the use of examples as elements of conviction. The types of empirical demonstration are:

 Naive empiricism: when in the approach of a conjecture and in the demonstration selected examples are used without any specific criteria. Two types of naive empiricisms are identified: Perceptual: when students rely on visual or tactile elements. Inductive: when students are based on mathematical elements or relationships detected in the example. Ο Crucial experiment: when the conjecture is demonstrated using an example that is chosen because it is presumed that in any other case it will give the same result. The following are identified.

Types of crucial experiments: Based on the example: when the students are based on the existence of a single example or in the absence of counterexamples for their demonstration. Constructive: when the students sustain their demonstrations in the constructions made on the example or in the way to get the example. Analytical: when selected examples are used and student demonstrations are based on properties and relationships observed in the example. Argumentation Processes through the use of Pragmatic Tests in an AGD with seventh grade students in seventh grade 2011 24 ο Generic example: when in the conjecture a specific example is used that is representative of a class.

 The three types of demonstration defined in the preceding paragraphs for the crucial experiment are presented in the generic examples.

 Deductive demonstrations: characterized by decontextualization. They are based on generic aspects of the problem, mental operations and logical deductions.

The types of deductive demonstrations are:

Mental experiment: when an example is used to help organize the demonstration. There are two types:

 Transformative mental experiment: when the demonstrations are based on mental operations that transform the initial problem into another equivalent. The example helps to foresee suitable transformations.

 Structural mental experiment: when the demonstrations are based on logical sequences derived from the problem data, the axioms, the accepted definitions or theorems. The example helps to organize or understand the steps of the deductions.

 Formal deduction: when the demonstration is based on mental operations, ie internalized actions on the mathematical object treated without the help of examples.

Pragmatic theories

The historical development of pragmatics has led to various explanations of partial aspects of the use of language complementary to each other. The various resulting theories refer only to partial aspects, so a complete study of all of them may require the analysis of several of these approaches. The theories about human communication most frequently cited in the grammar manuals are:

• Searle's speech acts theory, based on Austin's previous approach, which explains how certain misunderstandings occur and identifies some assumptions that explain why sentences with different grammatical forms have approximately the same pragmatic meaning.

• The theory of relevance of Sperber and Wilson, which through a semi-formal scheme explains how speakers make deductions and inferences from what is said in a conversation or linguistic interaction to create a linguistic context in which to interpret duly the following statements.

The principle of cooperation of Grice, which, in the style of game theory, studies how participants in an interaction use certain tacit principles that facilitate the inference and interpretation of what is said.

• The argumentation theory of Anscombre and Ducrot, which analyzes the linguistic elements associated with informal reasoning, observes how arguments and conclusions are introduced in a discussion and establishes what is a typical argument pragmatically relevant, although not strictly logical.

3.- ELEMENTS TO EVALUATE VOCABULARY AND GRAMMAR:

 EVALUATION CRITERIA:

Within the oral tests that can be performed this is one of the most controlled and therefore easier to correct, without thereby losing naturalness, since it is a situation that can occur in real life. The same activity can be adapted so that the student engages in a conversation with the examiner. In this way, the two oral skills, speaking and listening, would be integrated in the same activity. This option would only be recommended in the case of a small group of students.

Understanding of specific messages. The aim is to verify the ability to understand not only the overall idea but also specific details previously indicated, in simple oral and written texts and familiar to students, although other parts of the message are not accurately captured.

Global comprehension: The aim is to verify the students' ability to grasp in a general way the essentials in brief oral messages, issued in the most suitable communication conditions,

Understanding of specific messages. The aim is to verify the ability to understand not only the overall idea but also concrete details in simple oral and written texts familiar to the students, even if other parts of the message are not accurately captured.

Message production The ability of the students to express themselves orally in everyday and family situations is evaluated. Apologizing for possible pronunciation errors that do not affect comprehension.

 Pronunciation. The aim is to verify if the students have assimilated the phonological system of English, phonemes, rhythm and intonation, and if they are capable of using it both in the comprehension and in the production of simple, contextualized and familiar messages.

 Assimilation of the new vocabulary. This criterion is intended to assess the ability to understand and use appropriately vocabulary and lexical expressions object of learning. The assimilation of the lexicon will always be checked in contextualized situations familiar to the students.

Class participation. It is evaluated, by observing the behavior of the students, if they participate in a constructive way in the communication situations of the class, respecting the norms that allow the communicative exchange.

  Team work. It is evaluated, by observing the behavior of the students, if they participate in a constructive way in the activities of the group, collaborating with their work to the harmonic development of learning in the classroom.

Individual work. It is evaluated, by observing the behavior of the students, how they perform their individual work in terms of correcting the content and dedication in the presentation, as well as if they finish their work in the foreseen time.

 Interest to learn. It is intended to verify if the student shows interest in making progress in their learning and curiosity to know new things, if they pay attention in class, if they ask questions or if they ask questions.

 Respect for others. It is evaluated if the students show respect for their classmates, listening without interrupting, respecting the turn of the word and appreciating the ideas and opinions of others.

  Interest in knowing other cultures. It is evaluated if the students show interest in knowing cultural aspects of the English speaking peoples. If they pay attention when talking about these issues and if they ask questions to expand their knowledge.

  Use of forms of courtesy. It is intended to verify, through direct observation, if the students use the forms of courtesy they learn in English communication and incorporate them into the normal routine of the class.

Message production The ability of the students to express themselves orally in everyday and family situations is evaluated. Apologizing for possible pronunciation errors that do not affect comprehension.

 Pronunciation. The aim is to verify if the students have assimilated the phonological system of English, phonemes, rhythm and intonation, and if they are capable of using it both in the comprehension and in the production of simple messages.

Assimilation of the new vocabulary. The ability to understand and use appropriately the vocabulary and lexical expressions object of learning is assessed

Participation in the class. It is evaluated, by observing the behavior of the students, if they participate constructively in the communication situations of the class, respecting the norms that allow the communicative exchange.

Team work. if they participate constructively in the group's activities, collaborating with their work to the harmonious development of learning in the classroom.

Individual work. How they perform their individual work in terms of correctness in the content and dedication in the presentation, as well as if they finish their tasks in the foreseen time.

 Interest to learn. If the student shows interest in progressing in their learning and curiosity to know new things, if they pay attention in class, if they ask questions or if they ask questions.

Respect for others. It is evaluated if the students show respect for their classmates and teachers, listening without interrupting, respecting the turn of the word and appreciating the ideas and opinions of others.

Use of forms of courtesy. A, if the students use the forms of courtesy they learn in English communication and incorporate them into the normal routine of the class.

Grammar:

Written expression. As in the proposed activity to evaluate oral expression, in this case we suggest an exercise that forces the student to transmit a predefined content, which is given by the vignettes, but at the same time gives freedom to decide how to transmit it. For example, the following story is designed so that the student uses a specific vocabulary and concrete grammatical structures, the past tenses of the verb. In this way, those with a higher command of the language can produce more or less elaborated texts, with more or less complex sentences and more or less detailed descriptions. This summer you went on vacation to Spain. On your return you write a letter to a friend to tell him everything that happened to you.

Design short activities with easy-to-score test questions. It often happens that students with a high command of grammar have great difficulties when it comes to understanding oral texts. On the contrary, other students with mother tongues closer to Spanish, such as Portuguese or Italian, may have a lower level of grammar and yet be able to understand much more. If only production exercises are used this would benefit the first students; if on the contrary only the perception skills are evaluated, the second group of students would have many advantages. Therefore, we believe it is essential that reception and production tests be included so that both types of students have the opportunity to demonstrate their abilities.

In the existing manuals for teaching Spanish as a foreign language you can find numerous listening activities that include multiple choice questions and that can easily be included in a level test. These exercises respect the principles of the communicative approach, in fact they are commonly used in the classroom. It has come to our attention that these types of activities are usually included in progress tests but not as frequently in achievement tests. The exercise that we propose as an example, in addition to being brief, reflects a natural situation with which students can meet in real life. The test questions are designed in such a way that some require a simple comprehension of the message, while to answer others a more complete understanding is necessary.

exhaustive In this way the students of a lower level will be able to answer correctly some of the questions, although to respond to those of greater complexity a much more developed comprehension skill is required.

In the language class and in the progress tests, real or adapted texts are used very often as written comprehension activities followed by a series of questions. In the case of an achievement test it is possible to adapt these questions to the test or true / false formats. The activity that we propose below combines both types of questions and requires the student to put into practice the "scanning" and "skimming" reading techniques.

The teaching-learning process will be planned and directed from the needs of students who are systematically diagnosed, taking into account their interests and preferences, in terms of topics, types of activities and organizational forms of work in the classroom and out of this. In the treatment of grammatical structures it is essential to put the student in situations in which he can produce his own texts according to his communicative needs, and with a proper contextualization, so that the student knows how to use them, under what circumstances and for what purpose either through oral or written communication In this way the student produces independently and creatively in the writing of paragraphs, letters, notes, dialogues, monologues, etc. the closest thing to real life, motivated by communicative needs. To this principle, great importance is given to interaction in pairs or in small groups, since it involves implicit communication processes; read, empty of information, feedback and selection, which favor the cognitive development and support the bases for a more cooperated and effective learning. For this reason, it is necessary for teachers to use some techniques that help develop self-correction and self-control strategies in students.

4 Comprehensive tests to assess comprehension and production:

The development of reading, understood as reading ability, involves the development of a set of general and specific skills. These skills are interrelated and not necessarily hierarchical.

 Description Axis of Skills General Understanding This axis groups the skills that involve the consideration of a text as a whole, or from a global perspective. It implies the establishment of a hierarchy between the ideas and the selection of the most general and predominant ones. It also requires distinguishing between key ideas and secondary details. Obtaining Information In this axis we work with certain fragments of information. Readers should review, search, locate relevant information. It is about identifying the essential elements of a message: its character, the moment, the situation. They must collate the information provided in the question with literal or similar information in the text and use it to find the new information that is asked of them. Elaboration of an interpretation Requires readers to expand their initial impressions, logically processing the information so that they can get a more complete and specific understanding of what they have read.

Next, the specific skills associated with reading are presented.  General comprehension • Identify: main idea, theme, purpose of the text, general use of a text. Describe: people, objects and places. Explain: central theme. Infer: main idea, theme and / or purpose. Obtaining information Identify units of literal or equivalent information. Select units of literal or equivalent information. Infer meaning of a word or concept, change of the signifier. Elaboration of an interpretation Compare and contrast information. Relate information units. Deduce meaning of key lexicons, to interpret the content of a short text, using contextual clues and / or visual. Prepare or generate new information.

Knowledge of the language A. Lexical components • Basic vocabulary referring to the area of ​​daily life such as home, family, body, transportation, places, occupations, sports, food, meals, countries and nationalities, recreation, market, clothing . B. Morphosyntactic components. Verb To Be and other frequently used verbs; verb tenses simple and compound. Parts of the sentence: nouns, qualifying adjectives, possessive adjectives, personal pronouns, demonstrative pronouns, question adverbs. Structures for existence-non-existence, presence-absence, anteriority-after; quantity, time, duration. Structure of affirmative and negative sentences, formulation of questions, short answers. C. Textual sense • General elements of cohesion and concordance. Elements of daily communication: greeting, saying goodbye, introducing yourself and other people, asking about tastes and preferences, among others. It includes the use of authentic descriptive, narrative and instructive texts.

1.3.1. Levels of pragmatic analysis The functional pragmatic approach that serves as the basis for PREP and PREP-R aims to identify linguistic alterations and link them to their communicative effects. In this way, the basic elements of all communicative interaction (namely: sender, message, and receiver) are those that structure the structure of the instrument, giving rise to three levels of pragmatic analysis: enunciative, textual, and interactive. In the words of Gallardo Paúls (2009a: 58).

 Theoretical foundations 38 - Interactive pragmatics: these are pragmatic categories that arise when considering that every message is addressed to a receiver, so they focus on the taking of turns: agility of turn, conversational participation index, priority or sociolinguistic adjustment, etc. - Textual pragmatics: are categories specifically linked to the grammatical nature of the message that the speaker constructs, and to have such a message value must have cohesion (lexical adequacy and correferential chains, morphosyntactic structures, connectors) and coherence (textual superstructures).

 Enunciative pragmatics: these are categories that arise from the consideration of each statement as an intentional action on the part of a speaker, that is, speech acts and inferential meanings. " Thus, the level of enunciative pragmatics includes the categories related to the issuer perspective. These are notions from classical cognitive pragmatics, specifically, from the Speech Theory of Austin (1962) and Searle (1969), according to which the ability of the speaker to produce declarative, propositional speech acts is evaluated. and illocutionary; and of the Principle of Cooperation of Grice (1975), which allows us to assess its capacity to understand and convey implicit meanings.

At the level of textual pragmatics, the categories related to the message understood as text are examined, that is, as connected discourse. We know that currently the Discourse Analysis considers text all series of utterances issued with a communicative purpose for a specific receptor profile, and that exhibits coherence properties, whether oral or written. In this sense, the conversation is understood as a text of an oral and dialogical nature, that is, a text in which coherence and its corresponding cohesion linguistic marks (when they exist) are constructed collaboratively. Here we can observe categories such as the logical organization of content in certain conversational sequences and the management of thematic progression.

 The morphological, syntactic, and lexical-semantic elements necessary for cohesion are also examined. The detailed explanation of this level is given in section 4. Finally, the level of interactive pragmatics deals with the categories related to the receiver perspective, taken into account as possible next speaker. These are notions that come basically from Sacks' (1992) ethnomethodological Conversation Analysis, and which insist on the constant exchange of roles that takes place in the conversation, as well as its collaborative nature. Here we can see categories such as the theoretical foundations, adequate cooperation in the development of turn-taking and respect for alternation, the agility or rhythm of conversational exchange, the degree of active participation of the speaker, the interactional orientation of their turns, as well as as non-verbal parameters such as the use of gaze, facial expression, and non-compensatory gestures. Section 5 deals with the detailed description of this level.