9 Thou Shalt (Not): Further specific “do’s and don’ts”, with further examples
Author’s note: I intend to link these examples and counterexamples within the above sections.
9.1 Posing a good question
Avoid:
Vague questions
Questions that are hard to understand
Non-economic questions
Unanswerable questions
9.2 Organisation, structure, and focus
Integrate all components: You need to integrate your literature survey (and all other parts) into the full paper. You don’t want the parts to look “pasted in”.
Use a standard format
Don’t give excessive detail
Don’t stray from your topic
Focus on the economic question
Keep the reader’s attention
Don’t leave parts unfinished
9.3 Literature review, citations, and attribution
Plagiarism: Plagiarism is an academic crime. Plagiarism=theft! You wouldn’t steal a chicken, would you? So don’t steal an idea, please. (And in case you were thinking about stealing a chicken, please don’t do this either.)
Is this your own idea?: Be clear about what is yours and what comes form other sources.
From feedback on an essay: Is this your own finding? It looks like it came from somewhere else. If this is a direct quote, put quotation marks around it and reference the source. If it is a paraphrasing, you still should reference the source. If it is your own idea, be clear about this. Use a phrase like “according my analysis below…” or “I hypothesize that…” or “based on the findings of XXX and YYY, I hypothesize that”…
Reference missing
Figures from CNN show that there is a 10% decline in applicants and a 12% decline when excluding foreign applicants.
“Figures from CNN” is too general. Which figures do these refer to? You should give a proper and specific citation for an internet and news source (see guidelines below [give better example]). However, it would be even better to find the original source for this statistic (which should probably be mentioned in the CNN report).
In light of the problems of moral hazard discussed above, it has been suggested by many practitioners that the deployment of highly powered incentives can resolve the issues of free-riding.
Whenever you write something like “it has been suggested” you need to mention at least one author who has suggested this. As Wikipedia says, ‘citation needed’. (Also see the problem of Straw Men).
Referencing problems: Reference (citation) not formatted correctly
Software tools like Endnote and Zotero can help you with this.
Consider a recent immigrant to Britain; does their experience of the economy in their home country affect their beliefs about the UK economy or their risk preferences? (Smith and Jones, 2009)
“Consider:” In general when you give references you need to say something about what that article did. Did the cited article POSE this question? Did it provide evidence on this in one direction or the other? Did it contain a detailed discussion of measuring risk preferences? Be specific!
Better:
Smith and Jones (2009) considered whether the risk-preferences of immigrants to Britain and the USA were correlated to the degree of economic uncertainty in their home countries, using survey data collected using …
(You should give a brief description here of the data they used and their method.)
9.4 Logic and argumentation
Incompleteness: Don’t make an argument or state a result that leaves the reader confused. If it is important, fully explain the concept.
Thus the idea of team spirit should be promulgated by linking pay to the performance of the team as a whole rather than offering incentive pay on individual basis only which would serve as a stumbling block to the promotion of the team cohesiveness in the company.
Why would the individual-level performance pay harm cohesiveness? This needs to be followed up.
Circular logic (“A is true because A is true.”)
Higher income taxes discourage people from seeking work. This is because people will be less likely to seek work when income taxes are raised.
Picture from ‘Knots’ by R.D. Laing (Add citation details).
In a standard labor-supply model that assumes [key assumptions here], as in [give a reference to such a model here]… higher income taxes discourage people from seeking work.
Self contradiction
I argue that monopolistic firms are more likely to be innovative than competitive firms because barriers to entry will stop rival firms imitating one another. Firms in a competitive market need to compete with their rivals, with new products etc., while a monopoly has the safety of having a high total market share.
The first sentence seems to argue that monopolistic firms are more innovative, while the second sentence provides an argument for the opposite.
Getting it backwards
Whenever income taxes increase, people earn more per hour of work and thus they shift to working more hours.
Presumably, the author meant the opposite of this. Be careful not to get things “backwards”. Even if the reader can figure out what you meant, it makes the paper difficult to read.
Unfounded claims: What is the evidence?
The consequences of the recent recession in Japan are that households’ disposable income has decreased significantly. Many people do not have the luxuries that they once did and this is another reason why crime has increased, individuals still want the luxuries and possessions that they once had, despite not being able to afford them, and they will steal them in order to regain their status.
Several claims are made here without providing evidence or even references.
Spot the claims:
Disposable income significantly decreased, as a result of the crisis.
People are spending less on luxuries.
The lack of luxuries is leading people to commit crimes (theft in particular)
This is about “status”.
Think about what evidence could support these claims. Try to gather this evidence, or at least reliable sources making these points. If not, don’t make such strong claims, or state these as a “potential story” or “hypothesis.”
However, if theatre and screen actors were to decertify, this would essentially cause antitrust laws to be rendered void. This would cause aspects of the contractual agreements such as overtime limits, salary caps, etc. to be effectively dropped. This would cause a huge rise in elite actor salaries as well as a monumental increase in the power exerted over the industry by larger studios.
Exercise: try to spot the claims (unfold)
If actors were to decertify, this would cause antitrust laws to be voided.
If antitrust laws are rendered void, parts of the contract agreement will be dropped.
If these parts …
Unclear difference/distinction: If you say that two concepts are distinct, you need to show clearly explain why and how.
If GDP growth’s aim is not to strengthen “neoliberal” structures, politicians should strive to fit global labour and welfare objectives, such as the Rio accords.
(Also see the word “Dogma”)
As Smith et al, (2010) show in great detail, companies that lobbied the Atlantian Senate on predatory lending in the years prior to the crisis exhibited significantly more risk-taking, managed to pressure politicians into barring all new laws that would have prevented their profit-seeking recklessness and in the aftermath, exhibited a much higher chance of being bailed out by the government.
At the same time in Germany, after the infamous reforms of 2011, price was raided [raised?] to 5000-9000 per year which is significantly larger than in any other country of continental Europe.
The infamous Solow growth model is based on the simplistic premise that every economy must save a proportion of its national income in order to fund investment in capital goods.
Opinion:
In general, avoid giving your opinions without evidence or outside references. If you are giving an opinion, it needs to be stated as such.
Sadly, this is true.
Please don’t be sad!
As the shadow Mayor of Birmingham spoke in 2009 “The economic stagnation and cuts being imposed by the Liberal government inevitably create social tension.”
- Although this is given as a quote, the student wrote “as the shadow Mayor … spoke” indicating the author considers this point to be true.
Which words make the following statement opinionated?
With the Atlantian national debt increasing at extortionate rates, public finances were beginning to run out of control. ‘Estimates suggest that the total debt will rise from 57 per cent to 87 per cent of GDP by 2017’
How about here?
The answer, as John Smith, Minister of the Cabinet pointed out on January 1st in the Committee room on Public Policy, has so far been negative: “I don’t think we can ever reach this situation”. Even though he should be in favour of such engagements he is not
What is opinionated about the following?
Areas such as work-study and other work based education need to be fully used to increase the efficiency of the future labour market. Training and education is key in any form and the government can provide the funding and knowledge to do this
… or this one?…
Women’s participation in the labour market plays a key role in development. As women represent over half of society, it is necessary for both sexes to be involved equally in economic activities to satisfy economic needs.
Even opinions that might seem widely accepted still should not be stated as facts in a research context.
Critically examine sources
Don’t take everything that is in print (or written online) as gospel truth. Be skeptical and carefully evaluate the arguments and evidence presented. Try to really survey what has been written, to consider the range of opinions and the preponderance of the evidence. You also need to be careful to distinguish between “real research” and propaganda or press releases.
The returns to higher education in Atlantis are extremely high. For the majority of Atlanian students a university degree has increased their lifetime income by over 50%, as reported in the “Benefits of Higher Education” report put out by the Association of Atlantian Universities (2016).
This would seem to be a potentially biased source, as they would have a self-serving motive to inflate the estimates of the benefits of higher education.7
On the other hand, while it is okay to take into account the limitations of a model, it is not okay to harshly criticize it without providing careful explanation:
Smith (2014) found a return to education in Atlantis exceeding 50%. This result is unlikely to be true because the study was not a very good one.
Better:
However, this paper adopts a traditional control strategy. The author fails to consider unobservable characteristics that might drive both educational choices and lifetime income; this suggests the estimate may be biased upwards.
A standard econometrics reference (even from a textbook) to this well-known issue would also be helpful here, at least in a student assignment.
Non sequitur
A non sequitur is a statement in which the implication does not follow from the premise (or the evidence). Avoid this common mistake
If \(\rightarrow\) Then
He argues that if a corporation’s success is considered a public good, the presence of such goods in the supply chain can explain how an economy can function (although sub-optimally) with persistent failures of markets to deal with those public goods.
It is not clear how the presence of “such goods” (which ones, exactly?) explains the functioning of an economy.
Logical words: THEREFORE
University may not be a good investment for everyone as vocational careers and apprenticeships may be more suitable. Therefore it is difficult to measure the true effect that reduced government spending on higher education has on economic efficiency.
Here we see two reasonable statements (requiring more evidence and supporting arguments, of course, but leave this aside for now).
University may not be a good investment for everyone.
It may be difficult to measure the impact that reduced government spending on HE has on economic efficiency.
However, statement 2 does not follow from statement 1. The ‘therefore’ is misplaced. We need a better connection to justify a ‘therefore’ here.
Although this is an efficient way of not making the household increase the level of income taxation that is paid, the tax payer will still suffer indirectly. As the government decreases expenditure, public services feel the effect of this cutback. For example, since the Conservative government came into power in 1979, the police force has made several cut backs and job losses in order to save government expenditure, as it is a public good.
A logical leap
Competitive markets in theory are the most efficient form of markets. Although perfectly competitive markets rarely (if ever) exist, by Sweden implementing fees to international students, the Swedish higher education system became more competitive, thus more efficient.
Even if competitive markets are the “most efficient” form this does not imply that INCREASING the level of fees makes the market “more competitive”, nor that a “more competitive” market is more efficient.
Not economics:
Avoid discussing issues that are irrelevant to economics and also irrelevant to your paper.
Not “proven”:
“Proven” is a problematic word to use, particularly in the context of empirical analysis and statements pertaining to the real world. (See discussion of theory and empirical work in chapter 1.).
Through their economic model, they prove that as long as elites have incentives to invest in de facto power, through lobbying or corruption for example, they will invest as much as possible in order to gain favourable conditions in the future for their businesses.
A theoretical economic model can not really prove anything about the real world; they typically rely on strong simplifying assumptions.
As Smith et al proved using data from the 1910-1920 Scandanavian stock exchange, equity prices always increase in response to reductions in corporate tax rates.
Empirical work does not prove anything (nor does it claim to). It relies on statistical inference under specific assumptions, and an intuitive sense that evidence from one situation is likely to apply to other situations.
Straw men: If you are claiming to be rebutting a “common argument” or “conventional wisdom” you need to show that others have made the point that you are trying to rebut.
By the way, I’m including this cartoon because it makes a good example, not because I agree with the politics. (Personally, I love Obama and I thing the ACA was great.)
Moreover, most economists argue that monopolies and oligopolies will exist only if it is profitable to the public (otherwise it would be stringently regulated or dissolved), but the findings show that there is a very strong link between the persistence of an oligopoly and political manipulation.
This is not what most economists argue, at least not so broadly. The student should have given a reference to some author or textbook who makes this claim
Too strong claims
Too strong a claim to make; needs to be stated more cautiously.
Although this has a positive effect in helping in deficit reduction there are consequences that the economy has to face because of this. Robbery rates have increased tremendously, because of the lack of police on the streets of New Zealand a lower number of criminals are being caught and the economy is suffering.
Thirdly this essay is going to display why private volunteer efforts are more effective than a lump sum tax, for example because it reduces the incentive to work for private individuals of a society.
Tuition fees are charged by secondary schools to support with funding of staff and faculty and provide an efficient student learning experience.
Units?
What are the units of measurement here?
In Portugal, Italy and Greece we register a drop in fulltime likelihood respectively of around -0.144, - 0.119 and -0.161.
The likelihood of what? Is this a drop in percentage points, in probabilities, in proportion of the original probability, or what?
Vague, imprecise:** It is not clear what this is intended to mean.
This needs to be made clearer and more definite
This problem has many applications in real life, and this investigation will further explore this phenomenon as well as try to answer some questions regarding its relevance.
What sort of applications? What questions will you try to answer?
I use a model constructed from recent research, which deals with problems from earlier papers, I extend it to see the effects for different categories.
What is the recent research? What were the problems? In what sense does this paper extend it, and to what sorts of categories?
Confusing statements
He argues that encouraging competition among workers by linking rewards to each other’s accomplishments has no intrinsic value, rather a result of the effective use of information.
What is “intrinsic value”? The use of “rather” is confusing here. Rather than what? Perhaps the student meant something like:
He argues that linking a salesperson’s rewards to her peers’ does not directly encourage effort through a spirit of competition. However, when the employer can set incentives based on the overall peer group’s performance, this allows a more effective use of information and better targeted incentive schemes.
Smith et al (1899) hypothesize that when group members are given the authority to reward and sanction each other this led to an increase in group allocations and earnings whereas, though it was as a result of behaviour in previous rounds which did not survive recurrence.
This sentence has too many clauses. The “whereas” and “though” make it very difficult to interpret.
Define:
Please define all the concepts that you rely on if they are not standard or obvious. Try to define them in a way that is precise, relevant to your paper, and economically meaningful.
… that they had incorporated the concept of self-managing work teams in their organizational structure while 81% claimed to deploy employee participation groups in the year 1983.
Teamwork also helps companies to achieve more flexibility.
What is meant by “teamwork”? Putting several employees together on the same project? Offering compensation based on group (rather than individual) performance? And what is meant by “flexibility”? The ability for the company to adapt to changing markets by quickly producing new products? The ability to react to employees leaving and retiring unexpectedly?
Anecdotal
This evidence is merely anecdotal, involving a few specific examples. Consider whether more formal and general evidence would be more convincing.
Finland has a long history of churning out well-known musicians, politicians, innovators, and entrepreneurs, including such notables as Finn Skylzbz, a chemist who instituted the Imperial Prize, Svrkl Brzgkb, founder of TVWorld, a television store with locations around the globe, and Zyjzblr Jmplf , a social scientist, often called the Father of Sociology…
It could be argued that this is not just due to the high quality of education in Finland but that, up until very recently, the higher education in Finland was free.
More formal evidence could include overall educational and career outcomes for the Finnish population, overall levels of innovation, and comparisons between countries that do and do not offer free higher education. The anecdotes might have been a good start if it they were backed up by stronger evidence. However, by themselves these are not “stylized facts” that are strong or convincing enough to base a paper on.
9.5 Economic logic; avoiding fallacies; economic terminology
Price? : Shouldn’t this be reflected in the price?
Oil company shares are likely to be highly profitable in the future, because there have been many oil wells recently discovered.
The share prices should presumably have already risen to reflect the expected future profits (see “efficient markets”).
There is always a shortage of plumbers because few people want to work in an industry seen as “dirty”.
If plumbing were seen as undesirable work, there should be a “compensating differential” and plumbers should earn higher wages, attracting people to this field.
Cet. Par . (Ceteris paribus … all things being equal): Does this result or claim hold other things constant?
Comparative (versus absolute) advantage and the gains to trade
Raising prices does not always increase profit; firms will not ‘raise prices to make up for losses’
Small differences/changes will have small “marginal” effects
9.6 Statistical and econometric logic
What changed?: Basically, only a change can explain a change. If you are trying to explain an economic or social change, you can not merely cite a factor that was always present.
Medical doctors and insurers hold a great deal of political power. This explains why medical costs have dramatically increased in the United States in the last 20 years, relative to other countries, and relative to previous decades.
Why is this particular to the last 20 years? Why is it particular to the United States? If doctors always held a lot of power they might have already taken steps to increase fees even before the last 20 years. A more credible explanation would have involved the doctors and insurers power increasing the last 20 years. Alternately, something may have changed in the system that allowed them to better exploit this power.
Policymakers believe that lowering interest rates will reduce unemployment. However, lowering interest rates may also lead to inflation. This explains the high rates of inflation in the UK in recent years.
A part of this argument is missing: have policymakers only recently come around this point of view? Has there been more unemployment in recent years, leading to increased use of the ‘interest rate’ tool?
Kazakhstan’s trade deficit has been higher than that of other former Soviet politics. This is because the Soviet republics were largely dependent on Russia for their export links to the rest of the world.
If this were the case (and I have no idea if it is) why did it affect Kazakhstan in particular? Was Kazakhstan more dependent on Russia then the other Soviet republics?
There is a correct way to say “the evidence suggests that there are no large effects”, but this must be done carefully.
Do not confuse the lack of a result (due to limited statistical power) with a “finding of no effect”.
I analyzed data from four countries over six years. In my regressions with GDP as the dependent variable, found a large but insignificant coefficient on the prevalence of earthquakes. Therefore, I conclude that natural disasters have no impact on national income.
The null hypothesis was presumably “no effect of disasters on GDP.” If the student had found a significant effect he might have been able to reject this null hypothesis in favor of an alternative hypothesis (e.g., of a negative effect on GDP). However, failing to reject the null does not allow you to conclude that the null holds. In particular, in this case it is unlikely that the student’s very small data set would have allowed her to find a significant result, even if there were a strong impact of disasters on GDP in reality. We say “the test had low power” when it is unlikely to reject the null even when the alternative hypothesis is true.
Endogeneity
How is causality identified? How do you (or the cited paper) claim to identify a causal relationship?
This policy was explained by Smith and Johnson (2002) in their research on subsidies and redistribution in higher education. Their results showed that people with higher degree have higher salaries and so pay higher taxes. Thus subsidizing higher education leads to a large social gain.
The results the student discusses seem to show an association between higher degrees and higher salaries. The student seems to imply that the education itself led to higher salaries. This has not been shown by the cited paper. Perhaps people who were able to get into higher education would earn higher salaries anyway. There are ways economists used to try to identify a “causal effect” (by the way, this widely used term is redundant as all effects must have a cause), but a mere association between two variables is not enough.
As inflation was systematically lower during periods of recession, we see that too low a level of inflation increases unemployment.
Economists have long debated the nature of this “Phillips curve” relationship. There is much work trying to determine whether the association (to the extent it exists) is a causal one. We could not rule out reverse causality, or third factor that might cause changes in both variables.
9.7 Presentation of theory and equations
Label variables : All specified variables (in equations, tables, and charts) should be clearly defined and described.
I run the following regression:
\[\mathbf{\text{Wage}}_{\mathbf{i}}\mathbf{=}\mathbf{\beta}_{\mathbf{1}}\mathbf{+}\mathbf{\beta}_{\mathbf{2}}\mathbf{\text{ED}}_{\mathbf{i}}\mathbf{+}\mathbf{\beta}_{\mathbf{3}}\mathbf{\text{EX}}_{\mathbf{i}}\mathbf{+}\mathbf{\epsilon}_{\mathbf{i}}\]This equation is not clear. 1. Variables should be given brief but understandable names or abbreviations. 2. All variables should be explained. In empirical work, you should also fully explain the data source and construction of all the variables.
Aside: When writing code, I recommend specifying variable names (column names), as well as dataset file and folder names, and in general all objects in what is called lower_snake_case. E.g., rather than assigning a variable as something like gen Incomeperday = income/duration assign it as gen income_per_day = income/duration (or in R income_per_day <- income/duration)
Better:
\[\mathbf{\text{WAGE}}_{\mathbf{i}}\mathbf{=}\mathbf{\beta}_{\mathbf{1}}\mathbf{+}\mathbf{\beta}_{\mathbf{2}}\mathbf{\text{EDUCATION}}_{\mathbf{i}}\mathbf{+}\mathbf{\beta}_{\mathbf{3}}\mathbf{\text{EXPERIENCE}}_{\mathbf{i}}\mathbf{+}\mathbf{\epsilon}_{\mathbf{i}}\]
Where
EDUCATION: Years of post-primary education (in levels)
EXPERIENCE: Years of job experience (Age – years education – years unemployed)
9.8 Data presentation and presentation of results
The crimes:
Unlabeled variables
Confusing labels
Too many digits presented in the results
Results presented with the wrong scale
Not explaining the estimation technique
Not presenting important associated regression diagnostics and statistics
Presenting too many results (e.g., showing a set of dummies that could be hidden) or irrelevant regression diagnostics
Regression table in a nonstandard format. Worst-case: pasting in Stata output
Not including significance tests
Forgetting to put standard errors below coefficient estimates
Presenting the wrong significance tests
Problematic example (edit):
What are the units here for the wage? What is IMPPEN and for what years? What does each observation represent? Is this some measure of dollars per hour?
Note that the standard errors of the estimates seem to be wide; thus there is a lack of statistical power so the author should be careful not to claim to have found strong “evidence against” an effect.
For the 2SLS, what instrument(s) are used? Some diagnostic measures of the instrument should be given, testing whether it is “weak or strong”, doing an overidentification test if there are multiple instruments, and perhaps presenting the “first stage” regression(s).
Misinterpretation of coefficients
Listing “all results” in text
Not discussing the results sufficiently
Misleading figures and graphs
Wrong choice of graph type
Misleading graph scale
Not explaining the data source
Copying a figure without attribution
Problematic example II
Table 5: Key Variables in the Estimation
|
|
|
|
|
|
||
| Experience* |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|||||||
The table heading does not make much sense; the heading should say something about WHAT this regression is aiming to measure.
What is the dependent variable in this regression? Have all of the abbreviations been explained? What is the actual data source? The student did not need to mention the software used. The DF column is probably not needed.
The note about the quote indicates significant variables is not quite right. The student should specify what level of significance this indicates, (p <.05? p <.01?) And for which significance test. To be a bit pedantic, it also the COEFFICIENT and not the VARIABLE that may be significant.
Failing to explain and justify your technique
The gravity model is used by large numbers of researchers in their econometric applications, therefore I apply the gravity equation to model the bilateral flow of trade.
As noted in an earlier section, *justify* your approach, don’t just “blame others”.
Failing to explain and justify your assumptions, restrictions, or choice of instrumental variable
To overcome a possible endogeneity problem a two-stage least square approach is applied using suitable instruments for FDI.
What is the endogeneity problem? What instruments are used and how do you justify that they are “suitable” (the exclusion restriction).
Reviewing (suggestions for improvement); overall assessment and recommendations
Foc-qn: What is your focal question?
Rel-Lit: Read related literature
See Jnl: Look at related articles in peer-reviewed journals to see how to format or express this.
Oth-auth: See what other authors have written on this
Oth-opin: Consider other opinions and arguments on this point
Expln-Lit: Needs to better explain the cited literature
Relate-Lit: Needs to better relate the cited literature to this paper
Empl-Lit-Tcnq: Needs to better consider the empirical literature that answers this question and related questions, and explain the techniques used, and how these may or may not be applied to the current paper.
Another empirical study made by Smith et al.(2010) also found that the stringent labour standards for agricultural produce imposed by the European Union lead to significant losses for the African producers.
How did these authors do their analysis? What data and techniques did they use? Explain the precise nature and strength of their findings.*
Expln-Thy: Needs to explain theoretical models more clearly and formally
Thinking about it further… Perhaps the results the student discusses seem to show an association between higher degrees and higher salaries. The student seems to imply that the education itself led to higher salaries. This has not been shown by the cited paper. Perhaps people who were able to get into higher education would earn higher salaries anyway. There are ways economists used to try to identify a “causal effect” (by the way, this widely used term is redundant as all effects must have a cause), but a mere association between two variables is not enough.↩︎