Monthly Archives: February 2017

General selection: Is voluntary self-exclusion a good proxy measure for problem gambling?

A couple of months ago, Dr. Michael Auer and I published a short paper in the Journal of Addiction Medicine and Therapy (JAMT) critically addressing a recent approach by researchers that use voluntary self-exclusion (VSE) by gamblers as a proxy measure for problem gambling in their empirical studies. We argued that this approach is flawed and is unlikely to help in developing harm-minimization measures.

For those who don’t know, self-exclusion practices typically refer to the possibility for gamblers to voluntarily ban themselves from playing all (or a selection of) games over a predetermined period. The period of exclusion can typically be chosen by the gambler although some operators have non-negotiable self-exclusion periods. Self-exclusion in both online sites and offline venues has become an important responsible gambling practice that is widely used by socially responsible operators.

There are many reasons why players self-exclude. In a 2011 study in the Journal of Gambling Studies by Dr. Tobias Hayer and Dr. Gerhard Meyer, players frequently reported excluding as a preventive measure and annoyance with the gambling operator as reasons for VSE. Furthermore, only one-fifth of self-excluders reported to be problem gamblers (21.2%). A recent 2016 (conference) paper by Dr. Suzanne Lischer (2016) reported that in a study of three Swiss casinos, 29% of self-excluders were pathological gamblers, 33% were problem gamblers, and 38% were recreational gamblers. Given that many voluntary self-excluders do not exclude themselves for gambling-related problems, Dr. Lischer concluded that self-exclusion is not a good indicator of gambling-related problems. In line with these results, a 2015 study published in International Gambling Studies led by Simo Dragicevic compared self-excluders with other online players and reported no differences in the (i) mean number of gambling hours per month or (ii) minutes per gambling session. The study also reported that 25% of players self-excluded within one day of their registration with the online operator. This could also be due to the fact that online players can self-exclude with just a few mouse-clicks.

post-featured-image-glasgow

Most studies to date report that the majority of voluntary self-excluders tend to be non-problem gamblers. Additionally, in 2010, the Australian Productivity Commission reported 15,000 active voluntary self-exclusions from 2002 to 2009 and that this represented only 10-20% of the population of problem gamblers. This means that in addition to most self-excluders being non-problem gamblers, that most problem gamblers are not self-excluders. This leads to the conclusion that there is little overlap between problem gambling and self-excluding.

Over the decade, analytical approaches to harm minimization have become popular. This has led to the development of various tracking tools such as PlayScan (developed by Svenska Spel), Observer (developed by 888.com), and mentor (developed by neccton and myself). Furthermore, regulators are increasingly recognizing the importance of early risk detection via behavioural tracking systems. VSE also plays an important role in this context. However, some systems use VSE as a proxy of at-risk or problem gambling.

Based on the findings from empirical research, self-exclusion is a poor proxy measure for categorizing at-risk or problem gamblers and VSE should not be used in early problem gambling detection systems. The reasons for this are evident:

  • There is no evidence of a direct relationship between self-exclusion and problem gambling. As argued above, self-excluders are not necessarily problem gamblers and thus cannot be used for early risk detection.
  • There are various reasons for self-exclusion that have nothing to do with problem gambling. Players exclude for different reasons and one of the most salient appears to be annoyance and frustration with the operator (i.e., VSE is used as a way of venting their unhappiness with the operator). In this case, an early detection model based on self-exclusion would basically identify unhappy players and be more useful to the marketing department than to those interested in harm minimization
  • Problem gamblers who self-exclude are already actively changing their behaviour. The trans-theoretical ‘stages of change’ model (developed by Dr. Carlo DiClemente and Dr. James Prochaska) argues that behavioural change follows stages from pre-contemplation to action and maintenance. One could argue that the segment of players who self-exclude because they believe their gambling to be problematic are the ones who already past the stages where assistance is usually helpful in triggering action to cease gambling. These players are making use of a harm-minimization tool. The ones actually in need of detection and intervention are the ones who have not yet reached this stage of change yet and are not thinking about changing their behaviour at all. This is one more argument for the inappropriateness of self-exclusion as a proxy for problem gambling.

But what could be done to prevent the development of gambling-related problems in the first place? For the reasons outlined above, we would argue that the attempt to identify problem gambling via playing patterns that are derived from self-excluders does not assist harm minimization. Firstly, this approach does not target problem gamblers, and secondly it does not provide any insights into the prevention of such problems.

It is evident that any gambling environment should strive to minimize gambling-related harm and reduce the amount of gambling among vulnerable groups. It is also known that information that is given to individuals to enable behavioural change should encourage reflection because research has shown that self-monitoring can enable behavioural change in the desired direction. Dr. Jim Orford has also stated that attempts to explain such disparate gambling types from a single theoretical perspective are essentially a fool’s errand. This also complements the notion that problem gambling is not a homogenous phenomenon and there is not a single type of problem gambler (as I argued in my first book on gambling back in 1995). This also goes in line with the belief of Dr. Auer and myself that gambling sites have to personalize communication and offer the right player the right assistance based on their individual playing history. Recent research that Dr. Auer and I have carried out supports this line of thinking.

Studies have also shown that dynamic feedback in the form of pop-up messages has a positive effect on gambling behaviour and gambling-related thoughts. For instance, research from Dr. Michael Wohl’s team in Canada have found that animation-based information enhanced the effectiveness of a pop-up message related to gambling time limits. Our own research has found that an enhanced pop-up message (that included self-appraisal and normative feedback) led to significantly greater number of players ending their session than a simple pop-up message. In a real-world study of online gamblers, we also found that personalized feedback had a significant effect in reducing the time and money spent gambling.

Personalized feedback is a player-centric approach and in addition to gambling-specific research, there is evidence from many other areas that shows the beneficial effects on behavioural change. For instance, personalized messages have shown to enable behavioural change in areas such as smoking cessation, diabetes management, and fitness activity. Contrary to the self-exclusion oriented detection approach, we concluded in our recent JAMT paper that personalized feedback aims to prevent and minimize harm in the first place and is a much better approach to the prevention of problem gambling than using data from those that self-exclude from gambling.

Dr. Mark Griffiths, Professor of Behavioural Addiction, International Gaming Research Unit, Nottingham Trent University, Nottingham, UK

 Further reading

Auer, M. & Griffiths, M. D. (2014). Personalised feedback in the promotion of responsible gambling: A brief overview. Responsible Gambling Review, 1, 27-36.

Auer, M. Griffiths, M.D. (2015). The use of personalized behavioral feedback for online gamblers: an empirical study. Frontiers in Psychology, 6, 1406.  doi:10.3389/fpsyg.2015.01406

Auer, M., Griffiths, M.D. (2015). Testing normative and self-appraisal feedback in an online slot-machine pop-up in a real-world setting. Frontiers in Psychology. 6, 339 doi: 10.3389/fpsyg.2015.00339

Auer, M., Littler, A., Griffiths, M. D. (2015). Legal Aspects of Responsible Gaming Pre-commitment and Personal Feedback Initiatives. Gaming Law Review and Economics. 19, 444-456.

DiClemente, C. C., Prochaska, J. O., Fairhurst, S. K., Velicer, W. F., Velasquez, M. M., & Rossi, J. S. (1991). The process of smoking cessation: an analysis of precontemplation, contemplation, and preparation stages of change. Journal of Consulting and Clinical Psychology, 59, 295-304.

Dragicevic, S., Percy, C., Kudic, A., Parke, J. (2015). A descriptive analysis of demographic and behavioral data from Internet gamblers and those who self-exclude from online gambling platforms. Journal of Gambling Studies. 31, 105-132.

Gainsbury, S. (2013). Review of self-exclusion from gambling venues as an intervention for problem gambling. Journal of Gambling Studies, 30, 229-251.

Griffiths, M. D. (1995). Adolescent gambling. London: Routledge.

Griffiths, M.D. & Auer, M. (2016). Should voluntary self-exclusion by gamblers be used as a proxy measure for problem gambling? Journal of Addiction Medicine and Therapy, 2(2), 00019.

Hayer, T., & Meyer, G. (2011). Self-exclusion as a harm-minimization strategy: Evidence for the casino sector from selected European countries. Journal of Gambling Studies, 27, 685-700

Kim, H. S., Wohl, M. J., Stewart, M. K., Sztainert, T., Gainsbury, S. M. (2014). Limit your time, gamble responsibly: setting a time limit (via pop-up message) on an electronic gaming machine reduces time on device. International Gambling Studies, 14, 266-278.

Lischer, S. (2016, June). Gambling-related problems of self-excluders in Swiss casinos. Paper presented at the 16th International Conference on Gambling & Risk Taking, Las Vegas, USA.

Suurvali, H., Hodgins, D. C., Cunningham, J. A. (2010). Motivators for resolving or seeking help for gambling problems: A review of the empirical literature. Journal of Gambling Studies, 26, 1-33

Don’t blame the game: Parents, videogame content, and age ratings

Back in March 2015, BBC News reported that parents of children in 16 Cheshire county schools had been sent a letter saying that head teachers would report them to the authorities if they allowed their children to play videogames that are rated for adults (i.e., games that have an ‘18’ rating). The teachers claimed that popular games like Grand Theft Auto and Call of Duty are too violent to be played by those under the age of 18 years. They also stated that such games increased sexualised behaviour and left children vulnerable to sexual grooming. The schools also threatened to report parents who let their children play such games because it was a form of parental neglect. The author of the letter, Mary Hennessy Jones, was quoted as saying that:

“We are trying to help parents to keep their children as safe as possible in this digital era. It is so easy for children to end up in the wrong place and parents find it helpful to have some very clear guidelines”.

I’m sure the letter to parents was written with the best of intentions but as a parent of three ‘screenagers’ and someone that has spent almost three decades researching the effects of video games on human behaviour, this appears to be a very heavy-handed way to deal with the issue. Although it is illegal for any retailer to sell ‘18’ rated games to minors, it is not illegal for children to play such games, or illegal for parents to allow their children to play such games. Many parents need to be educated about the positives and negatives of playing video games but reporting them to the “authorities” is not the right way forward.

pegi_ratings_system

Back in the early 1990s I was probably the only academic in the UK carrying out scientific research on children’s video game playing. In fact, I was proud of my role in getting age ratings onto all video games in the first place, and for writing the text for educational information leaflets for parents (outlining the effects of excessive playing of such games) sponsored by the National Council for Educational Technology. There are many positive benefits of playing video games (something that I wrote about in a previous article for The Conversation).

I know from first-hand experience that children often play games that are age-inappropriate. Two years ago, my (then) 13-year old son said he was the only boy in his class that did not play or own the Call of Duty video game. This is also borne out by research evidence. One study that I was involved in found that almost two-thirds of children aged 11- to 13-years of age (63%) had played an 18+ video game. Unsurprisingly, boys (76%) were more likely than girls (49%) to have played an 18+ video game. Children were also asked about how often they played 18+ video games. Of the two-thirds who had played them, 8% reported playing them “all the time”, 22% reported playing them “most of the time”, 50% reported playing them “sometimes”, 18% reported playing them “hardly ever”. Again, boys were more likely than girls to play 18+ video games more frequently. Children were asked how they got access to 18+ plus video games. The majority had the games bought for them by family or friends (58%), played them at a friend’s house (35%), swapped them with friends (27%), or bought games themselves (5%). This research certainly appears to suggest that parents and siblings are complicit in the playing of age-inappropriate games.

There is a growing amount of scientific literature that has examined the content of video games designed for adults. For instance, a study led by Dr. Kimberley Thompson and published in the Archives of Pediatric and Adolescent Medicine attempted to quantify the depiction of violence, blood, sexual themes, profanity, substances, and gambling in adult (18+) video games and to assess whether the actual game content matched the content descriptor on the packaging. Although content descriptors for violence and blood provided a good indication of content in the 36 games examined, the authors concluded that 81% of the games studied (n=29) lacked content descriptors of other adult content. Other studies carried out by the same research team have found that adult content can be found in lots of games aimed at young children and teenagers.

Another study led by Dr. David Walsh published in Minerva Pediatrica tested the validity of media rating systems (including video games). Results showed that when the entertainment industry rated a product as inappropriate for children, parents also agreed that it was inappropriate. However, parents disagreed with many industry ratings that were designated as containing material as suitable for children. The products rated as appropriate for adolescents by the industry were of the greatest concern to parents.

The issue of children and adolescents playing 18+ games is no different from the debates about children and adolescents watching 18+ films. However, based on anecdotal evidence appears that parents are more likely to adhere to age ratings on films than they are on video games. This is one area that both media researchers and media educators need to inform parents to be more socially responsible in how they monitor their children’s leisure activity. A school sending out a threatening letter to parents is unlikely to change parental behaviour. Education and informed debate is likely to have a much greater effect in protecting our children from the potential harms of video game playing.

Dr. Mark Griffiths, Professor of Behavioural Addiction, International Gaming Research Unit, Nottingham Trent University, Nottingham, UK

 Further reading

Anderson, C.A., Gentile, D.A., & Dill, K.E. (2012). Prosocial, antisocial and other effects of recreational video games. In D.G. Singer, & J.L. Singer (Eds), Handbook of Children and the Media, Second Edition, (pp. 249-272). Thousand Oaks, CA: Sage.

Anderson, C. A., Shibuya, A., Ihori, N., Swing, E. L., Bushman, B.J., Sakamoto, A., Rothstein, H.R., & Saleem, M. (2010). Violent video game effects on aggression, empathy, and prosocial behavior in eastern and western countries: a meta-analytic review. Psychological Bulletin, 136, 151-173.

Bartlett, C. P., Anderson, C.A. & Swing, E.L. (2009). Video game effects confirmed, suspected and speculative: A review of the evidence. Simulation and Gaming, 40, 377-403.

Ferguson, C. J. (2007). Evidence for publication bias in video game violence effects literature: A meta analytic review. Aggression and Violent Behavior, 12, 470-482.

Ferguson, C. J. (2013). Violent video games and the supreme court: Lessons for the scientific community in the wake of Brown v. Entertainment Merchants Association. American Psychologists, 68, 57-74.

Ferguson, C. J., San Miguel, S. & Hartley, T. (2009).  Multivariate analysis of youth violence and aggression: The influence of family, peers, depression and media violence. Journal of Paediatrics, 155, 904-908.

Gentile, D. A. & Stone, W. (2005). Violent video game effects in children and adolescents: A review of the literature. Minerva Pediatrics, 57, 337-358.

Griffiths, M.D. (1998). Video games and aggression: A review of the literature. Aggression and Violent Behavior, 4, 203-212.

Griffiths, M.D. (2000). Video game violence and aggression: Comments on ‘Video game playing and its relations with aggressive and prosocial behaviour’ by O. Weigman and E.G.M. van Schie. British Journal of Social Psychology, 39, 147-149.

Griffiths, M.D. (2010). Age ratings on video games: Are the effective? Education and Health, 28, 65-67.

Griffiths, M.D. & McLean, L. (in press). Content effects: Online and offline games. In P. Roessler (Ed.), International Encyclopedia of Media Effects. Chichester: Wiley.

Grüsser, S.M., Thalemann, R. & Griffiths, M.D. (2007). Excessive computer game playing: Evidence for addiction and aggression?  CyberPsychology and Behavior, 10, 290-292.

Ivory, J.D., Colwell, J., Elson, M., Ferguson, C.J., Griffiths, M.D., Markey, P.M., Savage, J. & Williams, K.D. (2015). Manufacturing consensus in a divided field and blurring the line between the aggression concept and violent crime. Psychology of Popular Media Culture, 4, 222–229.

McLean, L. & Griffiths, M.D. (2013). The psychological effects of videogames on young people. Aloma: Revista de Psicologia, Ciències de l’Educació i de l’Esport, 31(1), 119-133.

McLean, L. & Griffiths, M.D. (2013). Violent video games and attitudes towards victims of crime: An empirical study among youth. International Journal of Cyber Behavior, Psychology and Learning, 2(3), 1-16.

Mehroof, M. & Griffiths, M.D. (2010). Online gaming addiction: The role of sensation seeking, self-control, neuroticism, aggression, state anxiety and trait anxiety. Cyberpsychology, Behavior, and Social Networking, 13, 313-316.

The words and the we’s: When is a new addiction scale not a new addiction scale?

“The words you use should be your own/Don’t plagiarize or take on loans/There’s always someone, somewhere/With a big nose, who knows” (Lyrics written by Morrissey from ‘Cemetry Gates’ (sic) by The Smiths)

Over the last few decades, research into ‘shopping addiction’ and ‘compulsive buying’ has greatly increased. In 2015, I along with my colleagues, developed and subsequently published (in the journal Frontiers in Psychology) a new scale to assess shopping addiction – the 7-item Bergen Shopping Addiction Scale (BSAS) which I wrote about in one of my previous blogs.

We noted in our Frontiers in Psychology paper that two scales had already been developed in the 2000s (i.e., one by Dr. George Christo and colleagues in 2003, and one by Dr. Nancy Ridgway and colleagues in 2008 – see ‘Further reading’ below), but that neither of these two instruments approached problematic shopping behaviour as an addiction in terms of core addiction criteria that are often used in the behavioural addiction field including salience, mood modification, tolerance, withdrawal, conflict, relapse, and problems. We also made the point that new Internet-related technologies have now greatly facilitated the emergence of problematic shopping behaviour because of factors such as accessibility, affordability, anonymity, convenience, and disinhibition, and that there was a need for a psychometrically robust instrument that assessed problematic shopping across all platforms (i.e., both online and offline). We concluded that the BSAS has good psychometrics, structure, content, convergent validity, and discriminative validity, and that researchers should consider using it in epidemiological studies and treatment settings concerning shopping addiction.

images

More recently, Srikant Amrut Manchiraju, Sadachar and Jessica Ridgway developed something they called the Compulsive Online Shopping Scale (COSS) in the International Journal of Mental Health and Addiction (IJMHA). Given that we had just developed a new shopping addiction scale that covered shopping across all media, we were interested to read about the new scale. The scale was a 28-item scale and was based on the 28 items included in the first step of BSAS development (i.e., initial 28-item pool). As the authors noted:

“First, to measure compulsive online shopping, we adopted the Bergen Shopping Addiction Scale (BSAS; Andreassen, 2015). The BSAS developed by Andreassen et al. (2015), was adapted for this study because it meets the addiction criteria (e.g., salience, mood modification, etc.) established in the DSM-5. In total, 28 items from the BSAS were modified to reflect compulsive online shopping. For example, the original item – ‘Shopping/buying is the most important thing in my life’ was modified as ‘Online shopping/buying is the most important thing in my life’… It is important to note that we are proposing a new behavioral addiction scale, specifically compulsive online shopping … In conclusion, the scale developed in this study demonstrated strong psychometric, structure, convergent, and discriminant validity, which is consistent with Andreassen et al.’s (2015) findings”.

Apart from the addition of the word ‘online’ to every item, all initial 28 items of the BSAS were used identically in the COSS. Therefore, I sought the opinion of several research colleagues about the ‘new’ scale. Nearly all were very surprised that an almost identical scale had been published. Some even questioned whether such wholescale use might constitute plagiarism (particularly as none of the developers of the COSS sought permission to adapt our scale).

According to the plagiarism.org website, several forms of plagiarism have been described including: “Copying so many words or ideas from a source that it makes up the majority of your work, whether you give credit or not” (p.1). Given the word-for-word reproduction of the 28 item–pool, an argument could be made that the COSS plagiarizes the BSAS, even though the authors acknowledge the source of their scale items. According to Katrina Korb’s 2012 article on adopting or adapting psychometric instruments:

“Adapting an instrument requires more substantial changes than adopting an instrument. In this situation, the researcher follows the general design of another instrument but adds items, removes items, and/or substantially changes the content of each item. Because adapting an instrument is similar to developing a new instrument, it is important that a researcher understands the key principles of developing an instrument…When adapting an instrument, the researcher should report the same information in the Instruments section as when adopting the instrument, but should also include what changes were made to the instrument and why” (p.1).

Dr. Manchiraju and his colleagues didn’t add or remove any of the original seven items, and did not substantially change the content of any of the 28 items on which the BSAS was based. They simply added the word ‘online’ to each existing item. Given that the BSAS was specifically developed to take into account the different ways in which people now shop and to include both online and offline shopping, there doesn’t seem to be a good rationale for developing an online version of the BSAS. Even if there was a good rationale, the scale could have made reference to the Bergen Shopping Addiction Scale in the name of the ‘new’ instrument. In a 2005 book chapter ‘Selected Ethical Issues Relevant to Test Adaptations’ by Dr. Thomas Oakland (2005), he noted the following in relation to plagiarism and psychometric test development:

Psychologists do not present portions of another’s work or data as their own, even if the other work or data source is cited … Plagiarism occurs commonly in test adaptation work (Oakland & Hu, 1991), especially when a test is adapted without the approval of its authors and publisher. Those who adapt a test by utilizing items from other tests without the approval of authors and publishers are likely to be violating ethical standards. This practice should not be condoned. Furthermore, this practice may violate laws in those countries that provide copyright protection to intellectual property. In terms of scale development, a measure that has the same original items with only one word added to each item (which only adds information on the context but does not change the meaning of the item) does not really constitute a new scale. They would find it really hard to demonstrate discriminant validity between the two measures”.

Again, according to Oakland’s description of plagiarism specifically in relation to the development of psychometric tests (rather than plagiarism more generally), the COSS appears to have plagiarized the BSAS particularly as Oakland makes specific reference to the adding of one word to each item (“In terms of scale development, a measure that has the same original items with only one word added to each item … does not really constitute a new scale”).

Still, it is important to point that I have no reason to think that this use of the BSAS was carried out maliciously. Indeed, it may well be that the only wrongdoing was lack of familiarity with the conventions of psychometric scale development. It may be that the authors took one line in our original Frontiers in Psychology paper too literally (the BSAS may be freely used by researchers in their future studies in this field”). However, the purpose of this sentence was to give fellow researchers permission to use the validated scale in their own studies and to avoid the inconvenience of having to request permission to use the BSAS and then waiting for an answer. Another important aspect here is that the BSAS (which may be freely used) consists of seven items only, not 28. The seven BSAS items were extracted from an initial item pool in accordance with our intent to create a brief shopping addiction scale. Consequently, there exists only one version of BSAS, the 7-item version. Here, Dr. Manchiraju and his colleagues seem to have misinterpreted this when referring to a 28-item BSAS.

(Please note: This blog is adapted using material from the following paper: Griffiths, M.D., Andreassen, C.S., Pallesen, S., Bilder, R.M., Torsheim, T. Aboujaoude, E.N. (2016). When is a new scale not a new scale? The case of the Bergen Shopping Addiction Scale and the Compulsive Online Shopping Scale. International Journal of Mental Health and Addiction, 14, 1107-1110).

Dr. Mark Griffiths, Professor of Behavioural Addiction, International Gaming Research Unit, Nottingham Trent University, Nottingham, UK

Further reading

Aboujaoude, E. (2014). Compulsive buying disorder: a review and update. Current Pharmaceutical Design, 20, 4021–4025.

Andreassen, C. S., Griffiths, M. D., Pallesen, S., Bilder, R. M., Torsheim, T., & Aboujaoude, E. (2015). The Bergen Shopping Addiction Scale: reliability and validity of a brief screening test. Frontiers in Psychology, 6, 1374. doi: 10.3389/fpsyg.2015.01374

Christo, G., Jones, S., Haylett, S., Stephenson, G., Lefever, R. M., & Lefever, R. (2003). The shorter PROMIS questionnaire: further validation of a tool for simultaneous assessment of multiple addictive behaviors. Addictive Behaviors, 28, 225–248.

Griffiths, M.D.  (2005). A ‘components’ model of addiction within a biopsychosocial framework. Journal of Substance Use, 10, 191-197.

Griffiths, M.D., Andreassen, C.S., Pallesen, S., Bilder, R.M., Torsheim, T. Aboujaoude, E.N. (2016). When is a new scale not a new scale? The case of the Bergen Shopping Addiction Scale and the Compulsive Online Shopping Scale. International Journal of Mental Health and Addiction, 14, 1107-1110.

Korb, K. (2012). Adopting or adapting an instrument. Retrieved September 12, 2016, from: http://korbedpsych.com/R09aAdopt.html

Manchiraju, S., Sadachar, A., & Ridgway, J. L. (2016). The Compulsive Online Shopping Scale (COSS): Development and Validation Using Panel Data. International Journal of Mental Health and Addiction, 1-15. doi: 10.1007/s11469-016-9662-6.

Maraz, A., Eisinger, A., Hende, Urbán, R., Paksi, B., Kun, B., Kökönyei, G., Griffiths, M.D. & Demetrovics, Z. (2015). Measuring compulsive buying behaviour: Psychometric validity of three different scales and prevalence in the general population and in shopping centres. Psychiatry Research, 225, 326–334.

Maraz, A., Griffiths, M. D., & Demetrovics, Z. (2016). The prevalence of compulsive buying in non-clinical populations: A systematic review and meta-analysis. Addiction, 111, 408-419.

Oakland, T. (2005). Selected ethical issues relevant to test adaptations. In Hambleton, R., Spielberger, C. & Meranda, P. (Eds.). Adapting educational and psychological tests for cross-cultural assessment (pp. 65-92). Mahwah, NY: Erlbaum Press.

Oakland, T., & Hu, S. (1991). Professionals who administer tests with children and youth: An international survey. Journal of Psychoeducational Assessment, 9(2), 108-120.

Plagiarism.org (2016). What is plagiarism? Retrieved September 12, 2016, from: http://www.plagiarism.org/plagiarism-101/what-is-plagiarism

Ridgway, N., Kukar-Kinney, M., & Monroe, K. (2008). An expanded conceptualization and a new measure of compulsive buying. Journal of Consumer Research, 35, 622–639.

Weinstein, A., Maraz, A., Griffiths, M.D., Lejoyeux, M. & Demetrovics, Z. (2016). Shopping addiction and compulsive buying: Features and characteristics of addiction. In V. Preedy (Ed.), The Neuropathology Of Drug Addictions And Substance Misuse (Vol. 3). (pp. 993-1008). London: Academic Press.