Revista Adolescência e Saúde

Revista Oficial do Núcleo de Estudos da Saúde do Adolescente / UERJ

NESA Publicação oficial
ISSN: 2177-5281 (Online)

Vol. 15 nº 4 - Oct/Dec - 2018

Original Article Imprimir 

Páginas 73 a 81


Reliability of instrument for evaluating the knowledge of adolescents about leprosy

Confiabilidad de instrumento para evaluación del conocimiento de adolescentes sobre hanseniasis

Confiabilidade de instrumento para avaliação do conhecimento de adolescentes sobre hanseníase

Autores: Alan Maique Ribeiro Fernandes da Costa1; Luiz Paulo de Jesus Miranda Silva2; Bruna Hinnah Borges Martins de Freitas3; Juliano Bortolini4

1. Graduation in Nursing by the Faculty of Nursing, Federal University of Mato Grosso (FAEN/UFMT). Cuiabá, MT, Brazil
2. Graduation in Nursing by the Faculty of Nursing, Federal University of Mato Grosso (FAEN/UFMT). Cuiabá, MT, Brazil
3. Graduation in Nursing by the Federal University of Mato Grosso (UFMT), Sinop campus. Master degree in Nursing by the Faculty of Nursing (FAEN/UFMT). Professor at the FAEN/UFMT. Cuiabá, MT, Brazil
4. Graduation in Mathematics by the Federal University of Mato Grosso (UFMT). Master degree and Doctorate in Agricultural Statistic and Experimentation of the Federal University of Lavras. Professor in the Department of Statistic at the UFMT. Cuiabá, MT, Brazil

Bruna Hinnah Borges Martins de Freitas
Faculdade de Enfermagem, Universidade Federal de Mato Grosso
Av. Fernando Corrêa da Costa, nº 2.367, Bairro Boa Esperança
Cuiabá, MT, Brasil. CEP: 78060-900
(bruhinnah@gmail.com)

PDF Portuguese      


Scielo

Medline


How to cite this article

Keywords: Psychometrics, reproducibility of results, knowledge, adolescent, leprosy.
Palabra Clave: Psicometría, reproducción de los test, conocimiento, adolescente, hanseniasis.
Descritores: Psicometria, reprodutibilidade dos testes, conhecimento, adolescente, hanseníase.

Abstract:
OBJECTIVE: Assess the reliability of the Instrument for the evaluation of adolescents' knowledge about leprosy.
METHODS: A methodological study with 150 adolescents with ages between 10 to 14 enrolled in public schools in Cuiabá - MT. Stability was analyzed using the Kappa coefficient and the internal consistency through the bisserial point correlation and Kuder-Richardson coefficient.
RESULTS: Regarding instrument stability, the majority of the items presented Kappa coefficient between 0.43 and 0.93, only one item obtained 0.27. As to the internal consistency, the exclusion of any of the instrument items did not increase the Kuder-Richardson coefficient of the general instrument, which was 0.82. All items presented the bisserial point correlation higher than 0.3.
CONCLUSION: It is concluded that the instrument in question seems reliable to evaluate the knowledge of adolescents about leprosy. The evaluation of the instrument in other environments and samples will allow the continuity of the validation process to better illustrate its applicability.

Resumen:
OBJETIVO: Evaluar la confiabilidad del Instrumento para la evaluación del conocimiento de adolescentes sobre hanseniasis.
MÉTODOS: Estudio metodológico realizado con 150 adolescentes de 10 a 14 años matriculados en escuelas públicas de Cuiabá - MT. La estabilidad se analizó mediante el coeficiente Kappa y consistencia interna por el punto biserialy del coeficiente de correlación Kuder -Richardson.
RESULTADOS: Con respecto a la estabilidad del instrumento, la mayoría de los artículos presentan el coeficiente Kappa de 0,43 a 0,93; sólo un elemento obtuvo 0,27. La consistencia interna, la exclusión de cualquiera de los elementos del instrumento no aumentó el valor del coeficiente de Kuder -Richardson del instrumento general, que fue de 0,82. Todos los artículos mostraron un punto biserial de correlación por encima de 0,3.
CONCLUSIÓN: Se concluye que el instrumento en cuestión parece confiable para evaluar el conocimiento de adolescentes sobre hanseniasis. La evaluación del instrumento en otros ambientes y muestras permitirá la continuidad del proceso de validación para mejor verificación de su aplicabilidad.

Resumo:
OBJETIVO: Avaliar a confiabilidade do Instrumento para avaliação do conhecimento de adolescentes sobre hanseníase.
MÉTODOS: Estudo metodológico realizado com 150 adolescentes de 10 a 14 anos matriculados em escolas públicas de Cuiabá - MT. A estabilidade foi analisada por meio do coeficiente Kappa e a consistência interna através da correlação do ponto bisserial e do coeficiente de Kuder-Richardson.
RESULTADOS: Em relação a estabilidade do instrumento, a maioria dos itens apresentou o coeficiente Kappa entre 0,43 a 0,93, apenas um item obteve 0,27. Quanto à consistência interna, a exclusão de qualquer um dos itens do instrumento não aumentou o valor do coeficiente de Kuder-Richardson do instrumento geral, que foi de 0,82. Todos os itens apresentaram a Correlação ponto bisserial superior a 0,3.
CONCLUSÃO: Conclui-se que o instrumento em questão parece confiável para avaliar o conhecimento de adolescentes sobre hanseníase. A avaliação do instrumento em outros ambientes e amostras permitirá a continuidade do processo de validação para melhor ilação de sua aplicabilidade.

INTRODUCTION

The leprosy is a chronic infectious and contagious condition with great potential to cause deformities and physical disabilities. Early diagnosis and timely treatment are elementary factors in the outcome of each case. The treatment is carried out through multi-drug therapy (MDT) offered by the Unified Health System (SUS)1.

The World Health Organization (WHO) in 1981, when seeing the control and eradication of leprosy, recommended the treatment with multidrug therapy and, the following years, instituted free treatment for people affected by leprosy1.

Through this policy, throughout the last decade the detection rate of leprosy in Brazil remained in a decreasing curve. However, the goal of the eradication of leprosy as a public health problem has not yet been achieved2. A hipper-endemic situation in children under 15 years old (≥10 cases per 100 thousand inhabitants) between the years 2001 to 2016 was observed in eight Units of the Brazilian Federation (UF), including Mato Grosso2. These findings suggest active transmission of the etiologic agent and failures in disease control strategies.

Considered a millenarian disease, leprosy is impregnated with stigmas resulting from a social construction that lasts until today, interfering significantly in its confrontation. The lack of knowledge about the disease allows the permanence of the myths and social conceptions that historically interfere in its course3.

For Bonin et al.4, the lack of knowledge about their state of health generates social isolation, biopsychosocial disorders and the increase of comorbidities These factors favor non-adherence to treatment, charging the public health service and interfering with the subject's own quality of life. In addition, knowledge contributes to the promotion of health and prevention of grievances. In this sense, health education is presented as an instrument to facilitate the construction of knowledge, since it promotes the opportunity to understand the aspects related to the critical and emancipator health-disease process, through deep reflection of the reality to which item is inserted5. Considering the vulnerability of adolescents in relation to leprosy, the need for an assertive and quality educational approach is highlighted.

This strategy was verified in the field of leprosy in nine studies, according to literature review on educational practices of leprosy with adolescents. These practices were implemented and researched mainly by nurses. But when it comes to knowledge, the evaluation of these interventions was not done with valid and reliable instruments6.

Therefore, although studies of educational interventions with evaluative component of the knowledge of adolescents and the need for valid and reliable for this evaluation instruments, these were not found in the literature that intends to examine for such constructs7.

In this sense, to evaluate the knowledge of adolescents about leprosy, Soares et al.10 built and validates an instrument as to the face, the contents and semantics. It obtains a Content Validity Index ( IVC) final from 0.89 for a committee of seven e specialists and has been validated semantically by 20 adolescents from 10 to 14 years old through the analysis technique brainstorm . He does not suggest changes in him, affirming good comprehension and absence of difficulties in its realization7.

Then, this instrument should be evaluated for reliability that, in accordance with Alexander et al.8 "It is a way of communicating or demonstrating the accuracy of the scientific process and the reliability of the information that describes how a particular test is shown and similar results are reproduced in different circumstances, indicating that nothing has changed." To assessment Reliability researcher can use stability test (test-retest) and internal consistency8.

The test-retest is that which makes performs judgment on the stability of the answers and check if the results are identical on different occasions. The measurement of this type of reliability highlights the susceptibility of the instrument to external factors over a period of time. The test-retest It is carried out using the same assessment questionnaire twice, and in the closing compares the scores8.9.

Polit and Beck9 defined as internal consistency the way as the questionnaire items are interrelated and measure the same construct of this. In this way, an evaluation device can be understood as internal consistency if it measures the same features and does not divert attention from it.

In this context, Bonin et al.4 establish that knowledge assessment instruments should be used in educational interventions in order to measure their impact, providing reliability to the construction of knowledge.

Thus, in order to develop educational intervention studies and measure their impact on adolescent knowledge about leprosy, it becomes necessary, in addition to the construction and validation in terms of face, content and semantics, already carried out by Soares et al.10 on the evaluation of the reliability of it. The determination of reliability is critical to ensuring the methodological quality of instruments measure 8. . In this way, the objective of this study was to evaluate the reliability of the Instrument for the Evaluation of Adolescent Knowledge on Leprosy (IACAH).


METHOD

This study is part of a matrix research entitled "Health education and active search for leprosy in children under fifteen in Cuiabá, MT" that was evaluated and approved for implementation by the Research Ethics Committee of the Júlio Muller University Hospital (HUJM), under the petitioner 1,579,925 and CAAE 53659616.5.00.005541, on June 8, 2016, in response to the recommendations of Resolution 466/2012 of the National Health Council concerning research related to human beings . The collection of data was initiated by authorization of the schools, signing of the term of free consent and clarified by the parents/responsible of the adolescents and term of agreement by the adolescents.

This is a methodological research study at the reliability of a psychometric instrument for data collection, already validated in terms of face , content and semantics. The study was conducted in six public state schools of the urban area of Cuiabá in Mato Grosso (MT) randomly selected, from February to May 2018.

For the reliability analysis, adolescents from 10 to 14 years of age enrolled in the selected schools, which attended the research and obtained the consent of the parents/caregivers, were selected for convenience. The sample was 150 subjects. There is no consensus in the literature about the reliability of the sample needed for the analysis, however, according to Sapnas and Zeller10, samples of at least 50 and at most 100 subjects are sufficient when desired evaluate the psychometric properties of the instruments of construct measurements .

The instruments used in the research were the Evaluation of knowledge of adolescents on leprosy (IACAH) and another for characterization of the study participants. Both instruments were developed and validated the face, content and semantics7.

The first one was elaborated containing 14 items and closed and multiple choice questions by means of nine domains defined by the researchers. The domains of the instrument are: definition and etiology, epidemiological facts, signs and symptoms, transmission, stigma and prejudice, diagnosis, treatment, deformities and physical disabilities and measures to control the disease. Content validation was carried out by a Committee of seven experts who obtained a Total Content Validity Index of 0,89 for the instrument. It was also validated semantically by 20 adolescents aged 10 to 14 years and is considered understandable and relevant in terms of what is proposed7.

The second instrument consists of 17 articles, with open and closed questions regarding the variables demographic and epidemiological. This achieves a content validity index of 0,93, which was considered by the understandable and relevant validation experts semantics7.

Variables demographic and epidemiological they studied were: age, gender (male and female), if they heard or if they receive information about Hansen 's disease (yes and no), what you have heard or received information about leprosy (at home, at school, in the unit of health or others), knows what is leprosy (yes and no), there were or there were cases of leprosy in the family (yes and no), it was already evaluated for leprosy (yes, no and I do not know ), situation in which he has been evaluated for leprosy, knows a neighbor who has or has had leprosy (yes and no), knows someone in his school who has or has had leprosy (yes and no), has or has had leprosy (yes and no), adolescent knowledge about leprosy (yes, insufficient, regular, good and optimal).

The questionnaires are self applicable and were delivered to adolescents in space offered by the school for data collection, being recollected by investigators after refilling. The application was carried out twice, called s test- and retest, with an interval of seven days between them, being the first or all participants in the study, and the second, with a sample of 30 subjects, as well as in another similar study11.

The structure of the database was made using the software Epiinfo 3.5.4 with independent double classification. Data from both banks were compared using the tool Data Compare to identify possible inconsistencies, and later they were corrected by consulting the original pigtail documents.

For the reliability analysis, internal consistency and test-retest were considered. In this study, it does not apply to equivalence that does not imply the consideration or judgment of the evaluators, which occurs in the interviews and observations8.

For both, the stability analysis was obtained through the test-retest of the instrument, with the coefficient kappa for the dichotomous and ordinal variables8. For the coefficient of Kappa used the following classification: if Kappa <0,0 the agreement is poor; if 0,0 <Kappa <0,20 the concordance is mild; 0,21 <Kappa <0,40 the concordance is just; 0,41 <Kappa <0,60 concordance is moderate; if 0,61 <Kappa <0,80 the agreement is substantial, and if 0,81 <Kappa <1,0 the concordance is almost perfect12. These coefficients evaluate whether a group of individuals agrees with one another in the evaluation of a construct or, more specifically for test-retest, if the same individual maintains its assessment of a construct, which is the desired.

The internal consistency was analyzed by calculating the coefficient of Kuder -Richardson, which is used to format the dichotomous response. In this case, it was considered if the person knows or not the answer for each element, and the degree of knowledge was classified by the number of correct answers8. This coefficient compares the elements of the questionnaire and specifies the average correlation between them. On this scale, it is possible to note that they range from 0,00 to 1,00 and 0,70 of the targeted value by some authors like minimum acceptable, 0,80 as good and above 0,90 is considered excellent8,9.

The correlation biserial point was used to determine the correlation between the variables and identify the concurrence of the article with the score of the instrument. It was considered the total score of the instrument of that individual. When the item has a good adhesion to the biserial measuring instrument, the correlation must be higher to 0,313.

In the process to fill the test, the hits were counted and classified according the concepts: insufficient (up to 24% of hits), regular (25% to 49% of hits), good (50% to 74% of hits) or optimum (75% to 100% hits)14. Statistical analyzes were performed in the R software. Numerical variables were described by descriptive statistics (mean, median and standard deviation) and categorical nominal variables were described or present in frequency tables.


RESULTS

The sample (n = 150) was constituted by adolescents from 10 to 14 years old enrolled in public schools in Cuiabá (MT). The average age of the participants was 12.07 years (SD: 1.18), being 95 (63.3%) of the female gender and 55 (36.7%) of the male gender. It is noteworthy that 92 (61.3%) of the adolescents already heard or received information about leprosy , being that the majority (n = 50, 54.3%) emphasizes that the school provided such knowledge.

Only 13 (8.8%) of the individuals reported having a history of leprosy in the family, eight (5.3%) had a history of leprosy in the neighbors and three (2.0%) history of leprosy in school colleagues. Among the participants, three (2.0%) had a history of leprosy.

It was found that 76 (51.7%) of the participants said they didn´t know what leprosy is. However, when evaluated on knowledge about leprosy, 57 (38.0%) were classified as two optimal, 41 (27.4%) as good, 38 (25.3%) and 14 as a regular (9, 3%), bad.

The psychometric properties of the Instrument for the Evaluation of the Knowledge of Adolescents on Leprosy (IACAH) were evaluated in the present study in terms of stability and internal consistency. With regard to stability, the results of the first application of the instrument and the results of the second retest were placed in tables. Regarding the concordance of the items, one presented fair agreement, two almost perfect, five moderate and six substantial, as presented in table 1.




The internal consistency of the instrument with its 14 items evaluated by the coefficient Kuder-Richardson was considered good and is shown in Table 2.




According can be seen in Table 3, there was a good adherence to the items of the instrument (correlation point biserial> 0.3) and the elimination of any of the elements of the instrument didn´t increase the value of the coefficient of general instrument Kuder -Richardson, maintaining, therefore, the 14 previously defined items.




DISCUSSION

To measure the knowledge of adolescents about leprosy, it is essential to have a valid and reliable instrument8. The results of this study suggest that the instrument developed and validated by Soares et al.7 or showed good applicability.

As for the reliability of the instrument, the stability of most of the articles or considered substantial almost perfectly in line with other research, with positive values and values considered statistically significants11.15. However, it is inferred that in the present study there were alterations in the responses obtained in the items that presented low values of concordance. The main problem with the test-retest approach is that construct as and l knowledge can be modified by experience that occurs between the field of application, regardless of the stability of l instrument9. Research with this age group is complex, due to the difficulty of adolescents in maintaining its own unique speech16.

Regarding internal consistency, a value considered good was identified, since high correlations were obtained between the items and the whole instrument, being within the expected reliability factor, which refers to the degree to which an instrument produces consistent and coherent results based on its scores17. A similar result was found in a study that aimed to evaluate a tool to verify the knowledge, experience and attitude of the professionals of Family Health in comparison with cases of child abuse, however, from other thematic18.

In the present study, although most of the participants mentioned not knowing what leprosy is, the knowledge of the study participants was, in general, considered optimal. However, a previous study conducted in Rio Grande do Norte with 109 adolescents aged 13 to 17 years showed that most adolescents have little knowledge19.

The results of this study may be related to adolescents and have heard or have received information about leprosy at some point in their life, with the school being targeted as the main place of contact with the subject. A review study integrates in educational practice of leprosy with teenagers, identified that all work included in the review were developed in schools6. The school environment stands out as the main scenario for the construction of knowledge in health, considering the context in which the adolescent is inserted to draw the most effective strategy in search of health promotion.

It is known that health education should be constituted by approaching subjects in community spaces, with the understanding of health as a social practice focused on the need of the population, valuing knowledge and previous developed knowledge. Therefore, the increase in information on leprosy may favor the construction of knowledge regarding the subject.

Ignorance of the disease is related to the low demand for health services in the face of signs and symptoms of the disease, difficulties in coping with the feeling of sadness and shame that result in late diagnosis, stigma and prejudice19,20. In this way, health education among adolescents is considered one of the components of leprosy control, since it contributes to the early diagnosis, timely treatment and demystification of false culturally constructed concepts.

The study is limited to the local reality and the environment of public education. It is suggested that the evaluation of the properties of the instrument be carried out in other research scenarios, due to the socio cultural variations existing between the Brazilian regions, expanding the number of participants in order to allow greater exploitation in terms of the reliability of the instrument. In addition, due to the lack of knowledge of the instrument that measures the variable studied, it was not possible to perform the criterion validation of the respective instrument.


CONCLUSION

It is concluded that the instrument in question seems reliable to evaluate the knowledge of adolescents about leprosy, since most of the items obtained stability between moderate and almost perfect and both the items, as well as the general instrument, showed good internal consistency. The evaluation of the instrument in other environments and samples will allow the continuity of the validation process to better understand its applicability.


NOTE

Article extracted from the course work "Reliability of an instrument for assessing the knowledge of adolescents about leprosy", presented to the Faculty of Nursing of the Federal University of Mato Grosso. Cuiabá-MT, Brazil.


REFERENCES

1. Rodrigues LC, Lockwood DNJ. Leprosy now: epidemiology, progress, challenges, and research gaps. Lancet Infect Dis. [Internet] 2011 [citado 2017 Nov 05];11:464-70. Disponível em: https://www.thelancet.com/pdfs/journals/laninf/PIIS1473-3099(11)70006-8.pdf

2. Schneider PB, Freitas BH. Tendência da hanseníase em menores de 15 anos no Brasil, 2001-2016. Cad saúde pública. [Internet] 2018 [citado 2018 Jun 17];34(3):1-11. Disponível em: http://www.scielo.br/pdf/csp/v34n3/1678-4464-csp-34-03-e00101817.pdf

3. Leite SCC, Sampaio CA, Caldeira AP. "Como ferrugem em lata velha": o discurso do estigma de pacientes institucionalizados em decorrência da hanseníase. Physis (Rio J.). [Internet] 2015 [citado 2017 Nov 20];25(1):121-138. Disponível em: http://www.scielo.br/pdf/physis/v25n1/0103-7331-physis-25-01-00121.pdf

4. Bonin CDB, Santos RZ, Ghisi GLM, Vieira AM, Benetti RAM. Construction and Validation of a Questionnaire about Heart Failure Patients' Knowledge of Their Disease. Arq Bras Cardiol. [Internet]. 2014[citado 2017 Out 15] 102(4):364-373. Disponível em: http://www.scielo.br/pdf/abc/v102n4/en_0066-782x-abc-20140032.pdf

5. Barbosa LB, Vasconcelos SML, Correia LOS, Ferreira RC. Nutrition knowledge assessment studies in adults: a systematic review. Rev Ciência & Saúde Col. [Internet]. 2016[citado 2018 Out 08] 21(2):449-462. Disponível em: http://www.scielo.br/pdf/csc/v21n2/en_1413-8123-csc-21-02-0449.pdf

6. Freitas BHBM, Silva FB, Jesus JMF, Alencastro MAB. Práticas educativas sobre hanseníase com adolescentes: revisão integrativa da literatura. Rev bras enferm. No prelo 2019;72(iss supl 2).

7. Soares JEF, Soares NLS, Freitas BHBM, Bortolini J. Validação de instrumento para avaliação do conhecimento de adolescentes sobre hanseníase. Acta Paul Enferm. 2018;31(5). No prelo.

8. Alexandre NMC, Gallasch CH, Lima MHM, Rodrigues RCM. A confiabilidade no desenvolvimento e avaliação de instrumentos de medida na área da saúde. Rev Eletr Enf. [Internet]. 2013[citado 2017 Out 10]15(3):802-9. Disponível em: https://www.fen.ufg.br/revista/v15/n3/pdf/v15n3a23.pdf

9. Polit DF, Beck CT. Fundamentos de Pesquisa em Enfermagem: avaliação de evidências para a prática da enfermagem. 7 ed. Porto Alegre: Artmed, 2011

10. Sapnas KG, Zeller RA. Minimizing sample size when using exploratory fator analysis for measurement. J Nurs Meas. 2002;10(2):135-154.

11. Jansen AC, Marziale MHP, Santos CB, Spadotti RA, Santos DMSS. Validação para o Brasil do comply with post-exposure management among health care workers. Rev Esc Enferm da USP. [Internet] 2016 [citado 2018 Jun 17];50(6):973-981. Disponível em: http://www.scielo.br/pdf/reeusp/v50n6/pt_0080-6234-reeusp-50-06-00973.pdf

12. Landis J, Koch G. The measurements of observer agreement for categorical data. Biometrics. 1977;33(1):159-74.

13. Vendramine CMM, Dias AS. Teoria de resposta ao item na análise de uma prova de estatística em universitários. Psico-USF. [Internet] 2005 [citado 2018 Jun 05];10(2):201-210. Disponível em: http://www.scielo.br/pdf/pusf/v10n2/v10n2a12.pdf

14. Silva FB, Gondim EC, Henrique NCP, Fonseca LMM, Mello DF. Educational intervention involving young mothers: gaining knowledge on childcare. Acta paul enferm. [Internet]. 2018 [citado 2018 Jul 04]; 31( 1 ): 32-8. Disponível em: http://www.scielo.br/pdf/ape/v31n1/en_0103-2100-ape-31-01-0032.pdf

15. Chaise FO, Kasten AP, Furlanetto TS, Pasa J, Candotti CT. O. Validade e reprodutibilidade do questionário de hábitos relacionados ao trabalho para trabalhadores do SAMU 192. Rev Terap Ocup Uni de São Paulo. [Internet] 2016 [citado 2018 Jun 17];27(2):199-215. Disponível em: http://www.revistas.usp.br/rto/article/view/100411/116479

16. Coutinho LG, Souza SN, Oliveira BO. Encontros e desencontros entre adolescência e educação: relato de pesquisa-intervenção. Rev Psicol. [Internet] 2012 [citado 2018 Jun 17]; 24(2):341-352 Disponível em: http://www.scielo.br/pdf/fractal/v24n2/a09v24n2.pdf

17. Bousso RS, Ichikawa CRF, Misko MD, Santos MR, Baliza MF, Mendes-Castillo AMC, Bianchi ARF. Validation of Family Management Measure for the Brazilian culture. Rev Bras Enferm [Internet]. 2017[citado 2018 Jun 17];70(6):1151-8. Disponível em: http://www.scielo.br/pdf/reben/v70n6/0034-7167-reben-70-06-1151.pdf

18. Silva-Oliveira F, Ferreira EF, Mattos FF, Ribeiro MTF, Cota LOM, Vale MP, et al. Adaptação transcultural e reprodutibilidade de questionário para avaliação de conhecimento e atitude de profissionais de saúde frente a casos de abuso físico infantil. Ciênc saúde coletiva [Internet]. 2014 Mar [citado 2018 July 04] ; 19( 3 ): 917-929. Disponível em: http://www.scielo.br/pdf/csc/v19n3/1413-8123-csc-19-03-00917.pdf

19. Monteiro BR, Pinheiro MGC, Isoldi DMR, Cabral AMF, Simpson CA, Mendes FRP. Hanseníase: enfocando a educação em saúde para o projovem. Rev pesqui cuid fundam [Internet] 2015 [citado 2018 Jun 10];7(supl.):49-55. Disponível em: http://seer.unirio.br/index.php/cuidadofundamental/article/view/5881/pdf_1

20. Loures LF, Mármora CHC, Barreto J, Duppre NC. Percepção do estigma e repercussões sociais em indivíduos com hanseníase. Psicol estud. [Internet] 2016 [citado 2018 Jun 17];21(4):665-675. Disponível em: http://www.redalyc.org/pdf/2871/287149565012.pdf
adolescencia adolescencia adolescencia
GN1 © 2004-2019 Revista Adolescência e Saúde. Fone: (21) 2868-8456 / 2868-8457
Núcleo de Estudos da Saúde do Adolescente - NESA - UERJ
Boulevard 28 de Setembro, 109 - Fundos - Pavilhão Floriano Stoffel - Vila Isabel, Rio de Janeiro, RJ. CEP: 20551-030.
E-mail: revista@adolescenciaesaude.com