Sains Malaysiana 52(3)(2023): 783-794

http://doi.org/10.17576/jsm-2023-5203-08

 

Meta-Analysis on Effect of Essential Oils and Extracts of Spices on the Microbiological Quality of Meat and Poultry Products

(Analisis Meta Kesan Minyak Pati dan Ekstrak Rempah ke atas Kualiti Mikrobiologi Produk Daging dan Poltri)

 

GITA ANGGITA1, FALEH SETIA BUDI2 & HARSI DEWANTARI KUSUMANINGRUM2

 

1Food Science Study Program, Department of Food Science and Technology, Faculty of Agricultural Engineering and Technology, IPB University, Bogor 16680 Indonesia

2Department of Food Science and Technology, Faculty of Agricultural Engineering and Technology, IPB University, Bogor 16680 Indonesia

 

Received: 23 July 2022/Accepted: 7 February 2023

 

Abstract

Different spices have been reported containing antimicrobial compounds that can prevent or reduce the microbiological spoilage or pathogenic bacteria in meat or poultry products with various results. This study evaluated the effect of essential oils and extracts of spices in reducing bacteria in meat/poultry products using a meta-analysis approach by comparing the Hedges'd effect size (standardized mean difference, SMD, and 95% confidence interval, CI). A total of 240 data, extracted from 10 articles that were selected by Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) from 121 full-text articles, were analyzed using Meta Essential tools. The results showed that essential oils had a better cumulative significant effect in reducing bacterial loads of meat/poultry products (SMD=-4.37, 95%CI=-5.63 to -3.10) in comparison to the extracts (SMD=-3.66, 95%Cl=-4.56 to -2.76). As essential oils, cassia (SMD=-58.17, 95%CI=-109.88 to -6.47) showed the best effect size, whereas as extract, ganghwayassuk (SMD=-4.19, 95%CI=-6.22 to -2.16) was the most significant. Furthermore, the total plate count was significantly affected by cassia (SMD=-58.17, 95%CI=-109.88 to -6.47), Enterobacteriaceae by sage (SMD=-5.93, 95%CI=-8.32 to -3.54), and coliform also by sage (SMD=-3.79, 95%CI=-6.76 to -0.82). In general, Salmonella spp. was found as pathogenic bacterium that was the most affected (SMD=-19.68 and 95%CI=-39.01 to -0.35). In the form of essential oils, dipping was the best way in reducing microorganisms, while as extracts, adding them in the products was the best method.  This study provided reliable data in selecting spices for applications to improve the quality and safety of meat and poultry products.

 

Keywords: Antimicrobial; meat/poultry-based products; meta-analysis; spices

 

AbstraK

Rempah yang berbeza telah dilaporkan mengandungi sebatian antimikrob yang boleh menghalang atau mengurangkan kerosakan mikrobiologi atau bakteria patogen dalam daging atau produk poltri dengan keputusan yang pelbagai. Kajian ini menilai kesan minyak pati dan ekstrak rempah dalam mengurangkan bakteria dalam produk daging/poltri menggunakan pendekatan analisis meta dengan membandingkan saiz kesan Hedges'd (perbezaan min standard, SMD dan selang keyakinan 95%, CI). Sebanyak 240 data yang diekstrak daripada 10 artikel yang dipilih melalui Item Pelaporan Pilihan untuk Ulasan Sistematik dan Analisis Meta (PRISMA) daripada 121 artikel teks penuh, telah dianalisis menggunakan alat Meta Perlu. Keputusan menunjukkan bahawa minyak pati mempunyai kesan ketara kumulatif yang lebih baik dalam mengurangkan beban bakteria produk daging/poltri (SMD=-4.37, 95%CI=-5.63 hingga -3.10) berbanding ekstrak (SMD=-3.66, 95% Cl=-4.56 hingga -2.76). Sebagai minyak pati, cassia (SMD=-58.17, 95%CI=-109.88 hingga -6.47) menunjukkan saiz kesan terbaik, manakala sebagai ekstrak, ganghwayassuk (SMD=-4.19, 95%CI=-6.22 hingga -2.16) adalah paling ketara. Tambahan pula, jumlah kiraan plat terjejas dengan ketara oleh cassia (SMD=-58.17, 95%CI=-109.88 hingga -6.47), Enterobacteriaceae oleh sej (SMD=-5.93, 95%CI=-8.32 hingga -3.54) dan koliform juga oleh sej (SMD=-3.79, 95%CI=-6.76 hingga -0.82). Secara umum, Salmonella spp. didapati sebagai bakteria patogen yang paling terjejas (SMD=-19.68 dan 95%CI=-39.01 hingga -0.35). Dalam bentuk minyak pati, pencelupan adalah cara terbaik dalam mengurangkan mikroorganisma, manakala sebagai ekstrak, menambahnya dalam produk adalah kaedah terbaik. Kajian ini menyediakan data yang boleh dipercayai dalam memilih rempah untuk digunakan dalam meningkatkan kualiti dan keselamatan produk daging dan poltri.

 

Kata kunci: Analisis meta; antimikrob; produk berasaskan daging/poltri; rempah

 

REFERENCES

Abdel-Wahab, M., El-Sohaimy, S.A., Ibrahim, H.A. & El-Makarem, H.S.A. 2020. Evaluation the efficacy of clove, sage, and kiwifruit peels extracts as natural preservatives for fish fingers. Annals of Agricultural Sciences 65(1): 98-106.

Afandi, F.A., Wijaya, C.H., Faridah, D.N., Suyatma, N.E. & Jayanegara, A. 2021. Evaluation of various starchy foods: A systematic review and meta-analysis on chemical properties affecting the glycemic index values based on in vitro and in vivo experiments. Foods 10(2): 1-28.

Amor, G., Porta, R., Sabbah, M., Caputo, L., Idbella, M., de Feo, V., Fechtali, T. & Mauriello, G. 2021. Basil essential oil: Composition, antimicrobial properties, and microencapsulation to produce active chitosan films for food packaging. Foods 10(121): 1-16.

Angelina, M., Turnip, M. & Khotimah, S. 2015. Antibacterial activity test of basil leaf ethanol extract (Ocimum sanctum L.) against Escherichia coli and Staphylococcus aureus bacterial growth. Protobiont 4(1): 184-189.

Ariyani, F., Setiawan, L.E. & Soetaredjo, F.E. 2008. Extraction of essential oils from lemongrass plants with methanol, acetone, and n-hexane solvents. Journal of Widya Engineering 7(2): 124-133.

Bortolotto, F.C.K., Ceccoti, S.P.C., Orso, P.B., Wolupeck, H.L., Holley, R.A., Luciano, F.B. & de Macedo, R.E.F. 2018. Combined effects of garlic essential oil and allyl isothiocyanate against Escherichia coli O157:H7 in vitro and in pork sausage. Ciência Rural 48(10): 1-7.

Cegiełka, A., Hać-Szymańczuk, E., Piwowarek, K., Dasiewicz, K., Słowiński, M. & Wrońska, K. 2019. The use of bioactive properties of sage preparations to improve the storage stability of low-pressure mechanically separated meat from chickens. Poultry Science 98(10): 5045-5053.

Cortés-Rojas, D.F., de Souza, C.R.F. & Oliveira, W.P. 2014. Clove (Syzygium aromaticum): A precious spice. Asian Pacific Journal of Tropical Biomedicine 4(2): 90-96.

Darna, Turnip, M. & Rahmawati. 2018. Identification of Enterobacteriaceae members in traditional foods of pangkong cuttlefish. Labora Medika Journal 2(2): 6-12.

de Melo, A.N.F., de Souza, G.T., Schaffner, D., de Oliveira, T.C.M., Maciel, J.F., de Souza, E.L. & Magnani, M. 2017. Changes in thermo-tolerance and survival under simulated gastrointestinal conditions of Salmonella enteritidis PT4 and Salmonella typhimurium PT4 in chicken breast meat after exposure to sequential stresses. International Journal of Food Microbiology 251: 15-23.

de Oliveira, M.M.M., Brugnera, D.F. & Piccoli, R.H. 2013. Essential oils of thyme and rosemary in the control of Listeria monocytogenes in raw beef. Brazilian Journal of Microbiology 44(4): 1181-1188.

Dewi, E.S., EL. Latifa, S., Fawwarahly. & Kautsar, R. 2016. Kualitas mikrobiologis daging unggas di RPA dan yang beredar di pasaran. Jurnal Ilmu Produksi dan Teknologi Hasil Peternakan 4(3): 379-385.

Fachriyah, E., Wibawa, P.J. & Awaliyah, A. 2020. Antibacterial activity of basil oil (Ocimum basilicum L.) and basil oil nanoemulsion. Journal of Physics Conference Series 1524(1): 1-11.

Gao, Y., Zhuang, H., Yeh, H.Y., Bowker, B. & Zhang, J. 2019. Effect of rosemary extract on microbial growth, pH, color, and lipid oxidation in cold plasma-processed ground chicken patties. Innovative Food Science and Emerging Technologies 57: 102168.

García-Díez, J., Alheiro, J., Pinto, A.L., Falco, V., Fraqueza, M.J. & Patarata, L. 2017. Synergistic activity of essential oils from herbs and spices used on meat products against food borne pathogens. Natural Product Communications 12(2): 281-286.

Ghorbani, A. & Esmaeilizadeh, M. 2017. Pharmacological properties of Salvia officinalis and its components. Journal of Traditional and Complementary Medicine 7(4): 433-440.

Govaris, A., Solomakos, N., Pexara, A. & Chatzopoulou, P.S. 2010. The antimicrobial effect of oregano essential oil, nisin and their combination against Salmonella enteritidis in minced sheep meat during refrigerated storage. International Journal of Food Microbiology 137(2-3): 175-180.

Gurevitch, J., Koricheva, J., Nakagawa, S. & Stewart, G. 2018. Meta-analysis and the science of research synthesis. Nature 555(7695): 175-182.

Haro-González, J.N., Castillo-Herrera, G.A., Martínez-Velázquez, M. & Espinosa-Andrews, H. 2021. Clove essential oil (Syzygium aromaticum L. myrtaceae): Extraction, chemical composition, food applications, and essential bioactivity for human health. Molecules 26(21): 6387.

Hernández-Ochoa, L., MacÍas-Castanãeda, C.A., Nevárez-Moorillón, G.V., Salas-Munãoz, E. & Sandoval-Salas, F. 2012. Antimicrobial activity of chitosan-based films including spices’ essential oils and functional extracts. CyTA - Journal of Food 10(2): 85-91.

Hernando, D., Septinova, D. & Adhianto, K. 2015. Water content and microbial quality of the meat in Bandar Lampung abattoirs. JIPT 3(1): 61-68.

Higgins, J.P.T. & Green, S. 2008. Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series. Chichester: John Wiley & Sons Ltd.

Huang, D.F., Xu, J.G., Liu, J.X., Zhang, H. & Hu, Q.P. 2014. Chemical constituents, antibacterial activity and mechanism of action of the essential oil from Cinnamomum cassia bark against four food-related bacteria. Microbiology 83(4): 357-365.

Hwang, K.E., Kim, H.W., Choi, Y.S., Lee, S.Y., Yeo, E.J., Ham, Y.K., Choi, S.M., Lee, M.A. & Kim, C.J. 2013. Evaluation of the antioxidant effect of ganghwayakssuk (Artemisia princes Pamp.) extract alone and in combination with ascorbic acid in raw chicken patties. Poultry Science 92(12): 3244-3250.

Hwang, K.E., Choi, Y.S., Choi, J.H., Kim, H.Y., Kim, H.W., Lee, M.A., Chung, H.K. & Kim, C.J. 2011. The antioxidative properties of ganghwayakssuk (Artemisia princeps Pamp.) extracts added to refrigerated raw chicken nugget batter against lipid oxidation. Korean Journal for Food Science of Animal Resources 31(2): 166-175.

Jaggi, R.K., Madaan, R. & Singh, B. 2003. Anticonvulsant potential of holy basil, Ocimum sanctum Linn., and its cultures. Indian Journal of Experimental Biology 41(11): 1329-1333.

Jahan, N., Naveed, S., Zeshan, M. & Tahir, M.A. 2016. How to conduct a systematic review: A narrative literature review. Cureus 8(11): 1-8.

Jayanegara, A., Wina, E. & Takahashi, J. 2014. Meta-analysis on methane mitigating properties of saponin-rich sources in the Rumen: Influence of addition levels and plant sources. Asian-Australasian Journal of Animal Sciences 27(10): 1426-1435.

Kacaniova, M., Galovicova, L., Valkova, V., Tvrda, E., Terentjeva, M., Ziarovska, J., Kunova, S., Savitskaya, T., Grinshpan, D., Stefanikova, J., Felsöciova, S., Vukovic, N. & Kowalczewski, P.L. 2021. Antimicrobial and antioxidant activities of Cinnamomum cassia essential oil and its application in food preservation. Open Chemistry 19(1): 214-227.

Kammon, A., Almaeyoufi, A. & Asheg, A. 2019. In vitro antimicrobial activity of clove oil against Gram negative bacteria isolated from chickens. Approaches in Poultry, Dairy & Veterinary Sciences 6(2): 542-546.

Karpin’ska-tymoszczyk, M. 2007. Effect of sage extract (Salvia officinalis L.) mixture of sage extract and sodium isoascorbate on the quality and shelf life of vacuum-packed turkey meatballs. Journal of Muscle Foods 18(4): 420-434.

Khanjari, A., Karabagias, I.K. & Kontominas, M.G. 2013. Combined effect of N, O-carboxymethyl chitosan and oregano essential oil to extend shelf life and control Listeria monocytogenes in raw chicken meat fillets. Food Science and Technology 53(1): 94-99.

Kumar, S., Mendiratta, S.K., Agrawal, R.K., Sharma, H. & Singh, B.P. 2018. Anti-oxidant and antimicrobial properties of mutton nuggets incorporated with blends of essential oils. Journal of Food Science and Technology 55(2): 821-832.

Kumudavally, K.V., Tabassum, A., Radhakrishna, K. & Bawa, A.S. 2011. Effect of ethanolic extract of clove on the keeping quality of fresh mutton during storage at ambient temperature (25±2 °C). Journal of Food Science and Technology 48(4): 466-471.

Macwan, S.R., Dabhi, B.K., Aparnathi, K.D. & Prajapati, J.B. 2016. Essential oils of herbs and spices: Their antimicrobial activity and application in preservation of food. International Journal of Current Microbiology and Applied Sciences 5(5): 885-901.

Martínez, L., Ros, G. & Nieto, G. 2020. Effect of natural extracts obtained from food industry by-products on nutritional quality and shelf life of chicken nuggets enriched with organic Zn and Se provided in broiler diet. Poultry Science 99(3): 1491-1501.

Mehdizadeh, T., Tajik, H., Langroodi, A.M., Molaei, R. & Mahmoudian, A. 2020. Chitosan-starch film containing pomegranate peel extract and Thymus kotschyanus essential oil can prolong the shelf life of beef. Meat Science 163: 108073.

Mittal, R., Kumar, R. & Chahal, H. 2018. Antimicrobial activity of Ocimum sanctum leaves extracts and oil. Journal of Drug Delivery and Therapeutics 8(6): 201-204.

Mitic-Culafic, D., Vukovic-Gacic, B., Knezevic-Vukcevic, J., Stankovic, S. & Simic, D. 2005. Comparative study on the antibacterial activity of volatiles from sage (Salvia officinalis L.). Archives of Biological Sciences 57(3): 173-178.

Moghimi, R., Aliahmadi, A., McClements, D.J. & Rafati, H. 2016. Investigations of the effectiveness of nanoemulsions from sage oil as antibacterial agents on some food borne pathogens. Food Science and Technology 71: 69-76.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., Altman, D., Antes, G., Atkins, D., Barbour, V., Barrowman, N., Berlin, J.A., Clark, J., Clarke, M., Cook, D., D'Amico, R., Deeks, J.J., Devereaux, P.J., Dickersin, K., Egger, M., Ernst, E., Gøtzsche, P.C., Grimshaw, J., Guyatt, G., Higgins, J., Ioannidis, J.P.A., Kleijnen, J., Lang, T., Magrini, N., McNamee, D., Moja, L., Mulrow, C., Napoli, M., Oxman, A., Pham, B., Rennie, D., Sampson, M., Schulz, K.F. Shekelle, P.G., Tovey, D. & Tugwell, P. 2009. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine 6(7): 1-6.

Mukhtarini. 2014. Extraction, separation of compounds, and identification of active compounds. Jurnal Kesehatan 8(2): 361-367.

Nugboon, K. & Intarapichet, K. 2015. Antioxidant and antibacterial activities of thai culinary herb and spice extracts, and application in pork meatballs. International Food Research Journal 22(5): 1788-1800.

Octavia, S. & Lan, R. 2014. The family Enterobacteriaceae. In The Prokaryotes.4th ed., edited by Rosenberg, E., DeLong, E.F., Lory, S., Stackebrandt, E. & Thompson, F. German: Springer Berlin.

Ouahida, B., Amina, B. & Halima, M.I. 2021. Antibacterial activity of sage leaves against pathogenic bacteria as affected by different drying temperature. Arabian Journal of Medicinal & Aromatic Plants 7(1): 74-92.

Palareti, G., Legnani, C., Cosmi, B., Antonucci, E., Erba, N., Poli, D., Testa, S. & Tosetto, A. 2016. Comparison between different D-Dimer cutoff values to assess the individual risk of recurrent venous thromboembolism: Analysis of results obtained in the DULCIS study. International Journal of Laboratory Hematology 38(1): 42-49.

Palupi, E., Jayanegara, A., Ploeger, A. & Kahl, J. 2012. Comparison of nutritional quality between conventional and organic dairy products: A meta-analysis. Journal of the Science of Food and Agriculture 92(14): 2774-2781.

Pattanayak, P., Behera, P., Das, D. & Panda, S. 2010. Ocimum sanctum Linn. A reservoir plant for therapeutic applications: An overview. Pharmacognosy Reviews 4(7): 95-105.

Pedrós-Garrido, S., Clemente, I., Calanche, J.B., Condón-Abanto, S., Beltrán, J.A., Lyng, J.G., Brunton, N., Bolton, D. & Whyte, P. 2020. Antimicrobial activity of natural compounds against Listeria spp. and their effects on sensory attributes in salmon (Salmo salar) and cod (Gadus morhua). Food Control 107: 106768.

Prasetya, N.B.A. & Ngadiwiyana, N. 2006. Identification of compounds compounding cinnamon bark oil (Cinnamomum cassia) using GC-MS. Journal of Scientific & Applied Chemistry 9(3): 81-83.

Sarıçoban, C. & Yilmaz, M.T. 2014. Effect of thyme cumin essential oils and butylated hydroxyl anisole butylated. Poultry Science 93: 456-463.

Semeniuc, C.A., Pop, C.R. & Rotar, A.M. 2017. Antibacterial activity and interactions of plant essential oil combinations against Gram-positive and Gram-negative bacteria. Journal of Food and Drug Analysis 25(2): 403-408.

Shan, B., Cai, Y.Z., Brooks, J.D. & Corke, H. 2009. Antibacterial and antioxidant effects of five spice and herb extracts as natural preservatives of raw pork. Journal of the Science of Food and Agriculture 89(11): 1879-1885.

Sharafati-Chaleshtori, R., Rokni, N., Rafieian-Kopaei, M., Drees, F., Sharafati-Chaleshtori, A. & Salehi, E. 2014. Use of Tarragon (Artemisia dracunculus) essential oil as a natural preservative in beef burger. Italian Journal of Food Science 26(4): 427-432.

Sharma, H., Mendiratta, S.K., Agrawal, R.K., Talukder, S. & Kumar, S. 2018. Studies on the potential application of various blends of essential oils as antioxidant and antimicrobial preservatives in emulsion based chicken sausages. British Food Journal 120(7): 1398-1411.

Sharma, H., Mendiratta, S.K., Agarwal, R.K., Kumar, S. & Soni, A. 2017. Evaluation of anti-oxidant and antimicrobial activity of various essential oils in fresh chicken sausages. Journal of Food Science and Technology 54(2): 279-292.

Sirocchi, V., Devlieghere, F., Peelman, N., Sagratini, G., Maggi, F., Vittori, S. & Ragaert, P. 2017. Effect of Rosmarinus officinalis L. essential oil combined with different packaging conditions to extend the shelf life of refrigerated beef meat. Food Chemistry 221: 1069-1076.

Šojić, B., Tomović, V., Savanović, J., Kocić-Tanackov, S., Pavlić, B., Jokanović, M., Milidrag, A., Martinović, A., Vujadinović, D. & Vukić, M. 2021. Sage (Salvia officinalis L.) essential oil as a potential replacement for sodium nitrite in dry fermented sausages. Processes 9(3): 1-15.

Sun, Q., Zhao, X., Chen, H., Zhang, C. & Kong, B. 2018. Impact of spice extracts on the formation of biogenic amines and the physicochemical, microbiological and sensory quality of dry sausage. Food Control 92: 190-200.

Tawfik, G.M., Dila, K.A.S., Mohamed, M.Y.F., Tam, D.N.H., Kien, N.D., Ahmed, A.M. & Huy, N.T. 2019. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Tropical Medicine and Health 47(1): 1-9.

Teixeira, B., Marques, A., Ramos, C., Serrano, C., Matos, O., Neng, N.R., Nogueira, J.M.F., Saraiva, J.A. & Nunes, M.L. 2013. Chemical composition and bioactivity of different oregano (Origanum vulgare) extracts and essential oil. Journal of the Science of Food and Agriculture 93(11): 2707-2714.

Thanissery, R. & Smith, D.P. 2014. Marinade with thyme and orange oils reduces Salmonella enteritidis and Campylobacter coli on inoculated broiler breast fillets and whole wings. Poultry Science 93(5): 1258-1262.

Toda, S. 2005. Antioxidative effects of polyphenols from leaves of Artemisia princeps Pamp. on lipid peroxidation in vitro. Journal of Food Biochemistry 29(3): 305-312.

Trendafilova, A., Moujir, L.M., Sousa, P.M.C. & Seca, A.M.L. 2021. Research advances on health effects of edible Artemisia species and some sesquiterpene lactones constituents. Foods 10(1): 1-38.

Triki, M., Herrero, A.M., Jiménez-Colmenero, F. & Ruiz-Capillas, C. 2013. Storage stability of low-fat sodium reduced fresh merguez sausage prepared with olive oil in konjac gel matrix. Meat Science 94(4): 438-446.

Umano, K., Hagi, Y., Nakahara, K., Shoji, A. & Shibamoto, T. 2000. Volatile chemicals identified in extracts from leaves of Japanese mugwort (Artemisia princeps Pamp.). Journal of Agricultural and Food Chemistry 48(8): 3463-3469.

USDA. 2021. Food Safety Information: Recalls & Public Health Alerts.  www.fsis.usda.gov/recalls.

Valdivieso-Ugarte, M., Gomez-Llorente, C., Plaza-Díaz, J. & Gil, Á. 2019. Antimicrobial, antioxidant, and immunomodulatory properties of essential oils: A systematic review. Nutrients 11(11): 1-29.

Van Haute, S., Raes, K., Devlieghere, F. & Sampers, I. 2017. Combined use of cinnamon essential oil and MAP/vacuum packaging to increase the microbial and sensorial shelf life of lean pork and salmon. Food Packaging and Shelf Life 12: 51-58.

Veličković, D.T., Randjelović, N.V., Ristić, M.S., Veličković, A.S. & Šmelcerović, A.A. 2003. Chemical constituents and antimicrobial activity of the ethanol extracts obtained from the flower, leaf and stem of Salvia officinalis L. Journal of the Serbian Chemical Society 68(1): 17-24.

Verloove, F. & Andeweg, R. 2020. Artemisia princeps L. (Asteraceae), an overlooked invasive Far Eastern weed in Western Europe. Gorteria 42(1): 1-18.

Vrinda, M.K. & Garg, S.R. 2001. Inhibitory effect of clove oil on Listeria monocytogenes in meat and cheese. Food Microbiology 18(6): 647-650.

Yamani, H.A., Pang, E.C., Mantri, N. & Deighton, M.A. 2016. Antimicrobial activity of Tulsi (Ocimum tenuiflorum) essential oil and their major constituents against three species of bacteria. Frontiers in Microbiology 7(681): 1-10.

Zhang, H., Wu, J. & Guo, X. 2016. Effects of antimicrobial and antioxidant activities of spice extracts on raw chicken meat quality. Food Science and Human Wellness 5(1): 39-48.

 

*Corresponding author; email: h_kusumaningrum@apps.ipb.ac.id

 

 

 

 

 

previous