how to write hypothesis for regression analysis

Save 10% on All AnalystPrep 2024 Study Packages with Coupon Code BLOG10 .

• Payment Plans
• Product List
• Partnerships

• Try Free Trial
• Study Packages
• Levels I, II & III Lifetime Package
• Video Lessons
• Study Notes
• Practice Questions
• Levels II & III Lifetime Package
• About the Exam
• About your Instructor
• Part I Study Packages
• Part I & Part II Lifetime Package
• Part II Study Packages
• Exams P & FM Lifetime Package
• Quantitative Questions
• Verbal Questions
• Data Insight Questions
• Live Tutoring
• About your Instructors
• EA Practice Questions
• Data Sufficiency Questions
• Integrated Reasoning Questions

Hypothesis Testing in Regression Analysis

Hypothesis testing is used to confirm if the estimated regression coefficients bear any statistical significance.  Either the confidence interval approach or the t-test approach can be used in hypothesis testing. In this section, we will explore the t-test approach.

The t-test Approach

The following are the steps followed in the performance of the t-test:

• Set the significance level for the test.
• Formulate the null and the alternative hypotheses.

$$t=\frac{\widehat{b_1}-b_1}{s_{\widehat{b_1}}}$$

\(b_1\) = True slope coefficient.

\(\widehat{b_1}\) = Point estimate for \(b_1\)

\(b_1 s_{\widehat{b_1\ }}\) = Standard error of the regression coefficient.

• Compare the absolute value of the t-statistic to the critical t-value (t_c). Reject the null hypothesis if the absolute value of the t-statistic is greater than the critical t-value i.e., \(t\ >\ +\ t_{critical}\ or\ t\ <\ –t_{\text{critical}}\).

Example: Hypothesis Testing of the Significance of Regression Coefficients

An analyst generates the following output from the regression analysis of inflation on unemployment:

$$\small{\begin{array}{llll}\hline{}& \textbf{Regression Statistics} &{}&{}\\ \hline{}& \text{Multiple R} & 0.8766 &{} \\ {}& \text{R Square} & 0.7684 &{} \\ {}& \text{Adjusted R Square} & 0.7394 & {}\\ {}& \text{Standard Error} & 0.0063 &{}\\ {}& \text{Observations} & 10 &{}\\ \hline {}& & & \\ \hline{} & \textbf{Coefficients} & \textbf{Standard Error} & \textbf{t-Stat}\\ \hline \text{Intercept} & 0.0710 & 0.0094 & 7.5160 \\\text{Forecast (Slope)} & -0.9041 & 0.1755 & -5.1516\\ \hline\end{array}}$$

At the 5% significant level, test the null hypothesis that the slope coefficient is significantly different from one, that is,

$$ H_{0}: b_{1} = 1\ vs. \ H_{a}: b_{1}≠1 $$

The calculated t-statistic, \(\text{t}=\frac{\widehat{b_{1}}-b_1}{\widehat{S_{b_{1}}}}\) is equal to:

$$\begin{align*}\text{t}& = \frac{-0.9041-1}{0.1755}\\& = -10.85\end{align*}$$

The critical two-tail t-values from the table with \(n-2=8\) degrees of freedom are:

$$\text{t}_{c}=±2.306$$

Notice that \(|t|>t_{c}\) i.e., (\(10.85>2.306\))

Therefore, we reject the null hypothesis and conclude that the estimated slope coefficient is statistically different from one.

Note that we used the confidence interval approach and arrived at the same conclusion.

Question Neeth Shinu, CFA, is forecasting price elasticity of supply for a certain product. Shinu uses the quantity of the product supplied for the past 5months as the dependent variable and the price per unit of the product as the independent variable. The regression results are shown below. $$\small{\begin{array}{lccccc}\hline \textbf{Regression Statistics} & & & & & \\ \hline \text{Multiple R} & 0.9971 & {}& {}&{}\\ \text{R Square} & 0.9941 & & & \\ \text{Adjusted R Square} & 0.9922 & & & & \\ \text{Standard Error} & 3.6515 & & & \\ \text{Observations} & 5 & & & \\ \hline {}& \textbf{Coefficients} & \textbf{Standard Error} & \textbf{t Stat} & \textbf{P-value}\\ \hline\text{Intercept} & -159 & 10.520 & (15.114) & 0.001\\ \text{Slope} & 0.26 & 0.012 & 22.517 & 0.000\\ \hline\end{array}}$$ Which of the following most likely reports the correct value of the t-statistic for the slope and most accurately evaluates its statistical significance with 95% confidence?     A. \(t=21.67\); slope is significantly different from zero.     B. \(t= 3.18\); slope is significantly different from zero.     C. \(t=22.57\); slope is not significantly different from zero. Solution The correct answer is A . The t-statistic is calculated using the formula: $$\text{t}=\frac{\widehat{b_{1}}-b_1}{\widehat{S_{b_{1}}}}$$ Where: \(b_{1}\) = True slope coefficient \(\widehat{b_{1}}\) = Point estimator for \(b_{1}\) \(\widehat{S_{b_{1}}}\) = Standard error of the regression coefficient $$\begin{align*}\text{t}&=\frac{0.26-0}{0.012}\\&=21.67\end{align*}$$ The critical two-tail t-values from the t-table with \(n-2 = 3\) degrees of freedom are: $$t_{c}=±3.18$$ Notice that \(|t|>t_{c}\) (i.e \(21.67>3.18\)). Therefore, the null hypothesis can be rejected. Further, we can conclude that the estimated slope coefficient is statistically different from zero.

Offered by AnalystPrep

Analysis of Variance (ANOVA)

Predicted value of a dependent variable, comparing probability and non-probabil ..., parametric and non-parametric test.

Parametric versus Non-parametric Tests of Independence A parametric test is a hypothesis test... Read More

Desirable Properties of an Estimator

A point estimator (PE) is a sample statistic used to estimate an unknown... Read More

Probability in Terms of Odds

Odds for and against an event represent a ratio of the desired outcomes... Read More

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Hypothesis Testing | A Step-by-Step Guide with Easy Examples

Published on November 8, 2019 by Rebecca Bevans . Revised on June 22, 2023.

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics . It is most often used by scientists to test specific predictions, called hypotheses, that arise from theories.

There are 5 main steps in hypothesis testing:

• State your research hypothesis as a null hypothesis and alternate hypothesis (H o ) and (H a  or H 1 ).
• Collect data in a way designed to test the hypothesis.
• Perform an appropriate statistical test .
• Decide whether to reject or fail to reject your null hypothesis.
• Present the findings in your results and discussion section.

Though the specific details might vary, the procedure you will use when testing a hypothesis will always follow some version of these steps.

Table of contents

Step 1: state your null and alternate hypothesis, step 2: collect data, step 3: perform a statistical test, step 4: decide whether to reject or fail to reject your null hypothesis, step 5: present your findings, other interesting articles, frequently asked questions about hypothesis testing.

After developing your initial research hypothesis (the prediction that you want to investigate), it is important to restate it as a null (H o ) and alternate (H a ) hypothesis so that you can test it mathematically.

The alternate hypothesis is usually your initial hypothesis that predicts a relationship between variables. The null hypothesis is a prediction of no relationship between the variables you are interested in.

• H 0 : Men are, on average, not taller than women. H a : Men are, on average, taller than women.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

For a statistical test to be valid , it is important to perform sampling and collect data in a way that is designed to test your hypothesis. If your data are not representative, then you cannot make statistical inferences about the population you are interested in.

There are a variety of statistical tests available, but they are all based on the comparison of within-group variance (how spread out the data is within a category) versus between-group variance (how different the categories are from one another).

If the between-group variance is large enough that there is little or no overlap between groups, then your statistical test will reflect that by showing a low p -value . This means it is unlikely that the differences between these groups came about by chance.

Alternatively, if there is high within-group variance and low between-group variance, then your statistical test will reflect that with a high p -value. This means it is likely that any difference you measure between groups is due to chance.

Your choice of statistical test will be based on the type of variables and the level of measurement of your collected data .

• an estimate of the difference in average height between the two groups.
• a p -value showing how likely you are to see this difference if the null hypothesis of no difference is true.

Based on the outcome of your statistical test, you will have to decide whether to reject or fail to reject your null hypothesis.

In most cases you will use the p -value generated by your statistical test to guide your decision. And in most cases, your predetermined level of significance for rejecting the null hypothesis will be 0.05 – that is, when there is a less than 5% chance that you would see these results if the null hypothesis were true.

In some cases, researchers choose a more conservative level of significance, such as 0.01 (1%). This minimizes the risk of incorrectly rejecting the null hypothesis ( Type I error ).

The results of hypothesis testing will be presented in the results and discussion sections of your research paper , dissertation or thesis .

In the results section you should give a brief summary of the data and a summary of the results of your statistical test (for example, the estimated difference between group means and associated p -value). In the discussion , you can discuss whether your initial hypothesis was supported by your results or not.

In the formal language of hypothesis testing, we talk about rejecting or failing to reject the null hypothesis. You will probably be asked to do this in your statistics assignments.

However, when presenting research results in academic papers we rarely talk this way. Instead, we go back to our alternate hypothesis (in this case, the hypothesis that men are on average taller than women) and state whether the result of our test did or did not support the alternate hypothesis.

If your null hypothesis was rejected, this result is interpreted as “supported the alternate hypothesis.”

These are superficial differences; you can see that they mean the same thing.

You might notice that we don’t say that we reject or fail to reject the alternate hypothesis . This is because hypothesis testing is not designed to prove or disprove anything. It is only designed to test whether a pattern we measure could have arisen spuriously, or by chance.

If we reject the null hypothesis based on our research (i.e., we find that it is unlikely that the pattern arose by chance), then we can say our test lends support to our hypothesis . But if the pattern does not pass our decision rule, meaning that it could have arisen by chance, then we say the test is inconsistent with our hypothesis .

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Descriptive statistics
• Measures of central tendency
• Correlation coefficient

Methodology

• Cluster sampling
• Stratified sampling
• Types of interviews
• Cohort study
• Thematic analysis

Research bias

• Implicit bias
• Cognitive bias
• Survivorship bias
• Availability heuristic
• Nonresponse bias
• Regression to the mean

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics. It is used by scientists to test specific predictions, called hypotheses , by calculating how likely it is that a pattern or relationship between variables could have arisen by chance.

A hypothesis states your predictions about what your research will find. It is a tentative answer to your research question that has not yet been tested. For some research projects, you might have to write several hypotheses that address different aspects of your research question.

A hypothesis is not just a guess — it should be based on existing theories and knowledge. It also has to be testable, which means you can support or refute it through scientific research methods (such as experiments, observations and statistical analysis of data).

Null and alternative hypotheses are used in statistical hypothesis testing . The null hypothesis of a test always predicts no effect or no relationship between variables, while the alternative hypothesis states your research prediction of an effect or relationship.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bevans, R. (2023, June 22). Hypothesis Testing | A Step-by-Step Guide with Easy Examples. Scribbr. Retrieved August 13, 2024, from https://www.scribbr.com/statistics/hypothesis-testing/

Is this article helpful?

Rebecca Bevans

Other students also liked, choosing the right statistical test | types & examples, understanding p values | definition and examples, what is your plagiarism score.

Linear regression - Hypothesis testing

by Marco Taboga , PhD

This lecture discusses how to perform tests of hypotheses about the coefficients of a linear regression model estimated by ordinary least squares (OLS).

Table of contents

Normal vs non-normal model

The linear regression model, matrix notation, tests of hypothesis in the normal linear regression model, test of a restriction on a single coefficient (t test), test of a set of linear restrictions (f test), tests based on maximum likelihood procedures (wald, lagrange multiplier, likelihood ratio), tests of hypothesis when the ols estimator is asymptotically normal, test of a restriction on a single coefficient (z test), test of a set of linear restrictions (chi-square test), learn more about regression analysis.

The lecture is divided in two parts:

in the first part, we discuss hypothesis testing in the normal linear regression model , in which the OLS estimator of the coefficients has a normal distribution conditional on the matrix of regressors;

in the second part, we show how to carry out hypothesis tests in linear regression analyses where the hypothesis of normality holds only in large samples (i.e., the OLS estimator can be proved to be asymptotically normal).

We also denote:

We now explain how to derive tests about the coefficients of the normal linear regression model.

It can be proved (see the lecture about the normal linear regression model ) that the assumption of conditional normality implies that:

How the acceptance region is determined depends not only on the desired size of the test , but also on whether the test is:

one-tailed (only one of the two things, i.e., either smaller or larger, is possible).

For more details on how to determine the acceptance region, see the glossary entry on critical values .

The F test is one-tailed .

A critical value in the right tail of the F distribution is chosen so as to achieve the desired size of the test.

Then, the null hypothesis is rejected if the F statistics is larger than the critical value.

In this section we explain how to perform hypothesis tests about the coefficients of a linear regression model when the OLS estimator is asymptotically normal.

As we have shown in the lecture on the properties of the OLS estimator , in several cases (i.e., under different sets of assumptions) it can be proved that:

These two properties are used to derive the asymptotic distribution of the test statistics used in hypothesis testing.

The test can be either one-tailed or two-tailed . The same comments made for the t-test apply here.

Like the F test, also the Chi-square test is usually one-tailed .

The desired size of the test is achieved by appropriately choosing a critical value in the right tail of the Chi-square distribution.

The null is rejected if the Chi-square statistics is larger than the critical value.

Want to learn more about regression analysis? Here are some suggestions:

R squared of a linear regression ;

Gauss-Markov theorem ;

Generalized Least Squares ;

Multicollinearity ;

Dummy variables ;

Selection of linear regression models

Partitioned regression ;

Ridge regression .

How to cite

Please cite as:

Taboga, Marco (2021). "Linear regression - Hypothesis testing", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-statistics/linear-regression-hypothesis-testing.

Most of the learning materials found on this website are now available in a traditional textbook format.

• F distribution
• Beta distribution
• Conditional probability
• Central Limit Theorem
• Binomial distribution
• Mean square convergence
• Delta method
• Almost sure convergence
• Mathematical tools
• Fundamentals of probability
• Probability distributions
• Asymptotic theory
• Fundamentals of statistics
• About Statlect
• Cookies, privacy and terms of use
• Loss function
• Almost sure
• Type I error
• Precision matrix
• Integrable variable
• To enhance your privacy,
• we removed the social buttons,
• but don't forget to share .

• Prompt Library
• DS/AI Trends
• Stats Tools
• Interview Questions
• Generative AI
• Machine Learning
• Deep Learning

Linear regression hypothesis testing: Concepts, Examples

In relation to machine learning , linear regression is defined as a predictive modeling technique that allows us to build a model which can help predict continuous response variables as a function of a linear combination of explanatory or predictor variables. While training linear regression models, we need to rely on hypothesis testing in relation to determining the relationship between the response and predictor variables. In the case of the linear regression model, two types of hypothesis testing are done. They are T-tests and F-tests . In other words, there are two types of statistics that are used to assess whether linear regression models exist representing response and predictor variables. They are t-statistics and f-statistics. As data scientists , it is of utmost importance to determine if linear regression is the correct choice of model for our particular problem and this can be done by performing hypothesis testing related to linear regression response and predictor variables. Many times, it is found that these concepts are not very clear with a lot many data scientists. In this blog post, we will discuss linear regression and hypothesis testing related to t-statistics and f-statistics . We will also provide an example to help illustrate how these concepts work.

Table of Contents

What are linear regression models?

A linear regression model can be defined as the function approximation that represents a continuous response variable as a function of one or more predictor variables. While building a linear regression model, the goal is to identify a linear equation that best predicts or models the relationship between the response or dependent variable and one or more predictor or independent variables.

There are two different kinds of linear regression models. They are as follows:

• Simple or Univariate linear regression models : These are linear regression models that are used to build a linear relationship between one response or dependent variable and one predictor or independent variable. The form of the equation that represents a simple linear regression model is Y=mX+b, where m is the coefficients of the predictor variable and b is bias. When considering the linear regression line, m represents the slope and b represents the intercept.
• Multiple or Multi-variate linear regression models : These are linear regression models that are used to build a linear relationship between one response or dependent variable and more than one predictor or independent variable. The form of the equation that represents a multiple linear regression model is Y=b0+b1X1+ b2X2 + … + bnXn, where bi represents the coefficients of the ith predictor variable. In this type of linear regression model, each predictor variable has its own coefficient that is used to calculate the predicted value of the response variable.

While training linear regression models, the requirement is to determine the coefficients which can result in the best-fitted linear regression line. The learning algorithm used to find the most appropriate coefficients is known as least squares regression . In the least-squares regression method, the coefficients are calculated using the least-squares error function. The main objective of this method is to minimize or reduce the sum of squared residuals between actual and predicted response values. The sum of squared residuals is also called the residual sum of squares (RSS). The outcome of executing the least-squares regression method is coefficients that minimize the linear regression cost function .

The residual e of the ith observation is represented as the following where [latex]Y_i[/latex] is the ith observation and [latex]\hat{Y_i}[/latex] is the prediction for ith observation or the value of response variable for ith observation.

[latex]e_i = Y_i – \hat{Y_i}[/latex]

The residual sum of squares can be represented as the following:

[latex]RSS = e_1^2 + e_2^2 + e_3^2 + … + e_n^2[/latex]

The least-squares method represents the algorithm that minimizes the above term, RSS.

Once the coefficients are determined, can it be claimed that these coefficients are the most appropriate ones for linear regression? The answer is no. After all, the coefficients are only the estimates and thus, there will be standard errors associated with each of the coefficients.  Recall that the standard error is used to calculate the confidence interval in which the mean value of the population parameter would exist. In other words, it represents the error of estimating a population parameter based on the sample data. The value of the standard error is calculated as the standard deviation of the sample divided by the square root of the sample size. The formula below represents the standard error of a mean.

[latex]SE(\mu) = \frac{\sigma}{\sqrt(N)}[/latex]

Thus, without analyzing aspects such as the standard error associated with the coefficients, it cannot be claimed that the linear regression coefficients are the most suitable ones without performing hypothesis testing. This is where hypothesis testing is needed . Before we get into why we need hypothesis testing with the linear regression model, let’s briefly learn about what is hypothesis testing?

Train a Multiple Linear Regression Model using R

Before getting into understanding the hypothesis testing concepts in relation to the linear regression model, let’s train a multi-variate or multiple linear regression model and print the summary output of the model which will be referred to, in the next section. 

The data used for creating a multi-linear regression model is BostonHousing which can be loaded in RStudioby installing mlbench package. The code is shown below:

install.packages(“mlbench”) library(mlbench) data(“BostonHousing”)

Once the data is loaded, the code shown below can be used to create the linear regression model.

attach(BostonHousing) BostonHousing.lm <- lm(log(medv) ~ crim + chas + rad + lstat) summary(BostonHousing.lm)

Executing the above command will result in the creation of a linear regression model with the response variable as medv and predictor variables as crim, chas, rad, and lstat. The following represents the details related to the response and predictor variables:

• log(medv) : Log of the median value of owner-occupied homes in USD 1000’s
• crim : Per capita crime rate by town
• chas : Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
• rad : Index of accessibility to radial highways
• lstat : Percentage of the lower status of the population

The following will be the output of the summary command that prints the details relating to the model including hypothesis testing details for coefficients (t-statistics) and the model as a whole (f-statistics) 

Hypothesis tests & Linear Regression Models

Hypothesis tests are the statistical procedure that is used to test a claim or assumption about the underlying distribution of a population based on the sample data. Here are key steps of doing hypothesis tests with linear regression models:

• Hypothesis formulation for T-tests: In the case of linear regression, the claim is made that there exists a relationship between response and predictor variables, and the claim is represented using the non-zero value of coefficients of predictor variables in the linear equation or regression model. This is formulated as an alternate hypothesis. Thus, the null hypothesis is set that there is no relationship between response and the predictor variables . Hence, the coefficients related to each of the predictor variables is equal to zero (0). So, if the linear regression model is Y = a0 + a1x1 + a2x2 + a3x3, then the null hypothesis for each test states that a1 = 0, a2 = 0, a3 = 0 etc. For all the predictor variables, individual hypothesis testing is done to determine whether the relationship between response and that particular predictor variable is statistically significant based on the sample data used for training the model. Thus, if there are, say, 5 features, there will be five hypothesis tests and each will have an associated null and alternate hypothesis.
• Hypothesis formulation for F-test : In addition, there is a hypothesis test done around the claim that there is a linear regression model representing the response variable and all the predictor variables. The null hypothesis is that the linear regression model does not exist . This essentially means that the value of all the coefficients is equal to zero. So, if the linear regression model is Y = a0 + a1x1 + a2x2 + a3x3, then the null hypothesis states that a1 = a2 = a3 = 0.
• F-statistics for testing hypothesis for linear regression model : F-test is used to test the null hypothesis that a linear regression model does not exist, representing the relationship between the response variable y and the predictor variables x1, x2, x3, x4 and x5. The null hypothesis can also be represented as x1 = x2 = x3 = x4 = x5 = 0. F-statistics is calculated as a function of sum of squares residuals for restricted regression (representing linear regression model with only intercept or bias and all the values of coefficients as zero) and sum of squares residuals for unrestricted regression (representing linear regression model). In the above diagram, note the value of f-statistics as 15.66 against the degrees of freedom as 5 and 194. 
• Evaluate t-statistics against the critical value/region : After calculating the value of t-statistics for each coefficient, it is now time to make a decision about whether to accept or reject the null hypothesis. In order for this decision to be made, one needs to set a significance level, which is also known as the alpha level. The significance level of 0.05 is usually set for rejecting the null hypothesis or otherwise. If the value of t-statistics fall in the critical region, the null hypothesis is rejected. Or, if the p-value comes out to be less than 0.05, the null hypothesis is rejected.
• Evaluate f-statistics against the critical value/region : The value of F-statistics and the p-value is evaluated for testing the null hypothesis that the linear regression model representing response and predictor variables does not exist. If the value of f-statistics is more than the critical value at the level of significance as 0.05, the null hypothesis is rejected. This means that the linear model exists with at least one valid coefficients. 
• Draw conclusions : The final step of hypothesis testing is to draw a conclusion by interpreting the results in terms of the original claim or hypothesis. If the null hypothesis of one or more predictor variables is rejected, it represents the fact that the relationship between the response and the predictor variable is not statistically significant based on the evidence or the sample data we used for training the model. Similarly, if the f-statistics value lies in the critical region and the value of the p-value is less than the alpha value usually set as 0.05, one can say that there exists a linear regression model.

Why hypothesis tests for linear regression models?

The reasons why we need to do hypothesis tests in case of a linear regression model are following:

• By creating the model, we are establishing a new truth (claims) about the relationship between response or dependent variable with one or more predictor or independent variables. In order to justify the truth, there are needed one or more tests. These tests can be termed as an act of testing the claim (or new truth) or in other words, hypothesis tests.
• One kind of test is required to test the relationship between response and each of the predictor variables (hence, T-tests)
• Another kind of test is required to test the linear regression model representation as a whole. This is called F-test.

While training linear regression models, hypothesis testing is done to determine whether the relationship between the response and each of the predictor variables is statistically significant or otherwise. The coefficients related to each of the predictor variables is determined. Then, individual hypothesis tests are done to determine whether the relationship between response and that particular predictor variable is statistically significant based on the sample data used for training the model. If at least one of the null hypotheses is rejected, it represents the fact that there exists no relationship between response and that particular predictor variable. T-statistics is used for performing the hypothesis testing because the standard deviation of the sampling distribution is unknown. The value of t-statistics is compared with the critical value from the t-distribution table in order to make a decision about whether to accept or reject the null hypothesis regarding the relationship between the response and predictor variables. If the value falls in the critical region, then the null hypothesis is rejected which means that there is no relationship between response and that predictor variable. In addition to T-tests, F-test is performed to test the null hypothesis that the linear regression model does not exist and that the value of all the coefficients is zero (0). Learn more about the linear regression and t-test in this blog – Linear regression t-test: formula, example .

Recent Posts

• K-Fold Cross Validation in Machine Learning – Python Example - August 16, 2024
• Gradient Boosting Machines (GBM): Concepts, Examples - August 16, 2024
• Random Forest Classifier – Sklearn Python Example - August 14, 2024

Ajitesh Kumar

One response.

Very informative

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

• Search for:
• Excellence Awaits: IITs, NITs & IIITs Journey

ChatGPT Prompts (250+)

• Generate Design Ideas for App
• Expand Feature Set of App
• Create a User Journey Map for App
• Generate Visual Design Ideas for App
• Generate a List of Competitors for App
• K-Fold Cross Validation in Machine Learning – Python Example
• Gradient Boosting Machines (GBM): Concepts, Examples
• Random Forest Classifier – Sklearn Python Example
• Decision Tree Regression vs Linear Regression: Differences
• Parametric vs Non-Parametric Models: Differences, Examples

Data Science / AI Trends

• • Prepend any arxiv.org link with talk2 to load the paper into a responsive chat application
• • Custom LLM and AI Agents (RAG) On Structured + Unstructured Data - AI Brain For Your Organization
• • Guides, papers, lecture, notebooks and resources for prompt engineering
• • Common tricks to make LLMs efficient and stable
• • Machine learning in finance

Free Online Tools

• Create Scatter Plots Online for your Excel Data
• Histogram / Frequency Distribution Creation Tool
• Online Pie Chart Maker Tool
• Z-test vs T-test Decision Tool
• Independent samples t-test calculator

Recent Comments

I found it very helpful. However the differences are not too understandable for me

Very Nice Explaination. Thankyiu very much,

in your case E respresent Member or Oraganization which include on e or more peers?

Such a informative post. Keep it up

Thank you....for your support. you given a good solution for me.

• Privacy Policy

Home » Regression Analysis – Methods, Types and Examples

Regression Analysis – Methods, Types and Examples

Table of Contents

Regression Analysis

Regression analysis is a set of statistical processes for estimating the relationships among variables . It includes many techniques for modeling and analyzing several variables when the focus is on the relationship between a dependent variable and one or more independent variables (or ‘predictors’).

Regression Analysis Methodology

Here is a general methodology for performing regression analysis:

• Define the research question: Clearly state the research question or hypothesis you want to investigate. Identify the dependent variable (also called the response variable or outcome variable) and the independent variables (also called predictor variables or explanatory variables) that you believe are related to the dependent variable.
• Collect data: Gather the data for the dependent variable and independent variables. Ensure that the data is relevant, accurate, and representative of the population or phenomenon you are studying.
• Explore the data: Perform exploratory data analysis to understand the characteristics of the data, identify any missing values or outliers, and assess the relationships between variables through scatter plots, histograms, or summary statistics.
• Choose the regression model: Select an appropriate regression model based on the nature of the variables and the research question. Common regression models include linear regression, multiple regression, logistic regression, polynomial regression, and time series regression, among others.
• Assess assumptions: Check the assumptions of the regression model. Some common assumptions include linearity (the relationship between variables is linear), independence of errors, homoscedasticity (constant variance of errors), and normality of errors. Violation of these assumptions may require additional steps or alternative models.
• Estimate the model: Use a suitable method to estimate the parameters of the regression model. The most common method is ordinary least squares (OLS), which minimizes the sum of squared differences between the observed and predicted values of the dependent variable.
• I nterpret the results: Analyze the estimated coefficients, p-values, confidence intervals, and goodness-of-fit measures (e.g., R-squared) to interpret the results. Determine the significance and direction of the relationships between the independent variables and the dependent variable.
• Evaluate model performance: Assess the overall performance of the regression model using appropriate measures, such as R-squared, adjusted R-squared, and root mean squared error (RMSE). These measures indicate how well the model fits the data and how much of the variation in the dependent variable is explained by the independent variables.
• Test assumptions and diagnose problems: Check the residuals (the differences between observed and predicted values) for any patterns or deviations from assumptions. Conduct diagnostic tests, such as examining residual plots, testing for multicollinearity among independent variables, and assessing heteroscedasticity or autocorrelation, if applicable.
• Make predictions and draw conclusions: Once you have a satisfactory model, use it to make predictions on new or unseen data. Draw conclusions based on the results of the analysis, considering the limitations and potential implications of the findings.

Types of Regression Analysis

Types of Regression Analysis are as follows:

Linear Regression

Linear regression is the most basic and widely used form of regression analysis. It models the linear relationship between a dependent variable and one or more independent variables. The goal is to find the best-fitting line that minimizes the sum of squared differences between observed and predicted values.

Multiple Regression

Multiple regression extends linear regression by incorporating two or more independent variables to predict the dependent variable. It allows for examining the simultaneous effects of multiple predictors on the outcome variable.

Polynomial Regression

Polynomial regression models non-linear relationships between variables by adding polynomial terms (e.g., squared or cubic terms) to the regression equation. It can capture curved or nonlinear patterns in the data.

Logistic Regression

Logistic regression is used when the dependent variable is binary or categorical. It models the probability of the occurrence of a certain event or outcome based on the independent variables. Logistic regression estimates the coefficients using the logistic function, which transforms the linear combination of predictors into a probability.

Ridge Regression and Lasso Regression

Ridge regression and Lasso regression are techniques used for addressing multicollinearity (high correlation between independent variables) and variable selection. Both methods introduce a penalty term to the regression equation to shrink or eliminate less important variables. Ridge regression uses L2 regularization, while Lasso regression uses L1 regularization.

Time Series Regression

Time series regression analyzes the relationship between a dependent variable and independent variables when the data is collected over time. It accounts for autocorrelation and trends in the data and is used in forecasting and studying temporal relationships.

Nonlinear Regression

Nonlinear regression models are used when the relationship between the dependent variable and independent variables is not linear. These models can take various functional forms and require estimation techniques different from those used in linear regression.

Poisson Regression

Poisson regression is employed when the dependent variable represents count data. It models the relationship between the independent variables and the expected count, assuming a Poisson distribution for the dependent variable.

Generalized Linear Models (GLM)

GLMs are a flexible class of regression models that extend the linear regression framework to handle different types of dependent variables, including binary, count, and continuous variables. GLMs incorporate various probability distributions and link functions.

Regression Analysis Formulas

Regression analysis involves estimating the parameters of a regression model to describe the relationship between the dependent variable (Y) and one or more independent variables (X). Here are the basic formulas for linear regression, multiple regression, and logistic regression:

Linear Regression:

Simple Linear Regression Model: Y = β0 + β1X + ε

Multiple Linear Regression Model: Y = β0 + β1X1 + β2X2 + … + βnXn + ε

In both formulas:

• Y represents the dependent variable (response variable).
• X represents the independent variable(s) (predictor variable(s)).
• β0, β1, β2, …, βn are the regression coefficients or parameters that need to be estimated.
• ε represents the error term or residual (the difference between the observed and predicted values).

Multiple Regression:

Multiple regression extends the concept of simple linear regression by including multiple independent variables.

Multiple Regression Model: Y = β0 + β1X1 + β2X2 + … + βnXn + ε

The formulas are similar to those in linear regression, with the addition of more independent variables.

Logistic Regression:

Logistic regression is used when the dependent variable is binary or categorical. The logistic regression model applies a logistic or sigmoid function to the linear combination of the independent variables.

Logistic Regression Model: p = 1 / (1 + e^-(β0 + β1X1 + β2X2 + … + βnXn))

In the formula:

• p represents the probability of the event occurring (e.g., the probability of success or belonging to a certain category).
• X1, X2, …, Xn represent the independent variables.
• e is the base of the natural logarithm.

The logistic function ensures that the predicted probabilities lie between 0 and 1, allowing for binary classification.

Regression Analysis Examples

Regression Analysis Examples are as follows:

• Stock Market Prediction: Regression analysis can be used to predict stock prices based on various factors such as historical prices, trading volume, news sentiment, and economic indicators. Traders and investors can use this analysis to make informed decisions about buying or selling stocks.
• Demand Forecasting: In retail and e-commerce, real-time It can help forecast demand for products. By analyzing historical sales data along with real-time data such as website traffic, promotional activities, and market trends, businesses can adjust their inventory levels and production schedules to meet customer demand more effectively.
• Energy Load Forecasting: Utility companies often use real-time regression analysis to forecast electricity demand. By analyzing historical energy consumption data, weather conditions, and other relevant factors, they can predict future energy loads. This information helps them optimize power generation and distribution, ensuring a stable and efficient energy supply.
• Online Advertising Performance: It can be used to assess the performance of online advertising campaigns. By analyzing real-time data on ad impressions, click-through rates, conversion rates, and other metrics, advertisers can adjust their targeting, messaging, and ad placement strategies to maximize their return on investment.
• Predictive Maintenance: Regression analysis can be applied to predict equipment failures or maintenance needs. By continuously monitoring sensor data from machines or vehicles, regression models can identify patterns or anomalies that indicate potential failures. This enables proactive maintenance, reducing downtime and optimizing maintenance schedules.
• Financial Risk Assessment: Real-time regression analysis can help financial institutions assess the risk associated with lending or investment decisions. By analyzing real-time data on factors such as borrower financials, market conditions, and macroeconomic indicators, regression models can estimate the likelihood of default or assess the risk-return tradeoff for investment portfolios.

Importance of Regression Analysis

Importance of Regression Analysis is as follows:

• Relationship Identification: Regression analysis helps in identifying and quantifying the relationship between a dependent variable and one or more independent variables. It allows us to determine how changes in independent variables impact the dependent variable. This information is crucial for decision-making, planning, and forecasting.
• Prediction and Forecasting: Regression analysis enables us to make predictions and forecasts based on the relationships identified. By estimating the values of the dependent variable using known values of independent variables, regression models can provide valuable insights into future outcomes. This is particularly useful in business, economics, finance, and other fields where forecasting is vital for planning and strategy development.
• Causality Assessment: While correlation does not imply causation, regression analysis provides a framework for assessing causality by considering the direction and strength of the relationship between variables. It allows researchers to control for other factors and assess the impact of a specific independent variable on the dependent variable. This helps in determining the causal effect and identifying significant factors that influence outcomes.
• Model Building and Variable Selection: Regression analysis aids in model building by determining the most appropriate functional form of the relationship between variables. It helps researchers select relevant independent variables and eliminate irrelevant ones, reducing complexity and improving model accuracy. This process is crucial for creating robust and interpretable models.
• Hypothesis Testing: Regression analysis provides a statistical framework for hypothesis testing. Researchers can test the significance of individual coefficients, assess the overall model fit, and determine if the relationship between variables is statistically significant. This allows for rigorous analysis and validation of research hypotheses.
• Policy Evaluation and Decision-Making: Regression analysis plays a vital role in policy evaluation and decision-making processes. By analyzing historical data, researchers can evaluate the effectiveness of policy interventions and identify the key factors contributing to certain outcomes. This information helps policymakers make informed decisions, allocate resources effectively, and optimize policy implementation.
• Risk Assessment and Control: Regression analysis can be used for risk assessment and control purposes. By analyzing historical data, organizations can identify risk factors and develop models that predict the likelihood of certain outcomes, such as defaults, accidents, or failures. This enables proactive risk management, allowing organizations to take preventive measures and mitigate potential risks.

When to Use Regression Analysis

• Prediction : Regression analysis is often employed to predict the value of the dependent variable based on the values of independent variables. For example, you might use regression to predict sales based on advertising expenditure, or to predict a student’s academic performance based on variables like study time, attendance, and previous grades.
• Relationship analysis: Regression can help determine the strength and direction of the relationship between variables. It can be used to examine whether there is a linear association between variables, identify which independent variables have a significant impact on the dependent variable, and quantify the magnitude of those effects.
• Causal inference: Regression analysis can be used to explore cause-and-effect relationships by controlling for other variables. For example, in a medical study, you might use regression to determine the impact of a specific treatment while accounting for other factors like age, gender, and lifestyle.
• Forecasting : Regression models can be utilized to forecast future trends or outcomes. By fitting a regression model to historical data, you can make predictions about future values of the dependent variable based on changes in the independent variables.
• Model evaluation: Regression analysis can be used to evaluate the performance of a model or test the significance of variables. You can assess how well the model fits the data, determine if additional variables improve the model’s predictive power, or test the statistical significance of coefficients.
• Data exploration : Regression analysis can help uncover patterns and insights in the data. By examining the relationships between variables, you can gain a deeper understanding of the data set and identify potential patterns, outliers, or influential observations.

Applications of Regression Analysis

Here are some common applications of regression analysis:

• Economic Forecasting: Regression analysis is frequently employed in economics to forecast variables such as GDP growth, inflation rates, or stock market performance. By analyzing historical data and identifying the underlying relationships, economists can make predictions about future economic conditions.
• Financial Analysis: Regression analysis plays a crucial role in financial analysis, such as predicting stock prices or evaluating the impact of financial factors on company performance. It helps analysts understand how variables like interest rates, company earnings, or market indices influence financial outcomes.
• Marketing Research: Regression analysis helps marketers understand consumer behavior and make data-driven decisions. It can be used to predict sales based on advertising expenditures, pricing strategies, or demographic variables. Regression models provide insights into which marketing efforts are most effective and help optimize marketing campaigns.
• Health Sciences: Regression analysis is extensively used in medical research and public health studies. It helps examine the relationship between risk factors and health outcomes, such as the impact of smoking on lung cancer or the relationship between diet and heart disease. Regression analysis also helps in predicting health outcomes based on various factors like age, genetic markers, or lifestyle choices.
• Social Sciences: Regression analysis is widely used in social sciences like sociology, psychology, and education research. Researchers can investigate the impact of variables like income, education level, or social factors on various outcomes such as crime rates, academic performance, or job satisfaction.
• Operations Research: Regression analysis is applied in operations research to optimize processes and improve efficiency. For example, it can be used to predict demand based on historical sales data, determine the factors influencing production output, or optimize supply chain logistics.
• Environmental Studies: Regression analysis helps in understanding and predicting environmental phenomena. It can be used to analyze the impact of factors like temperature, pollution levels, or land use patterns on phenomena such as species diversity, water quality, or climate change.
• Sports Analytics: Regression analysis is increasingly used in sports analytics to gain insights into player performance, team strategies, and game outcomes. It helps analyze the relationship between various factors like player statistics, coaching strategies, or environmental conditions and their impact on game outcomes.

Advantages and Disadvantages of Regression Analysis

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Cluster Analysis – Types, Methods and Examples

Inferential Statistics – Types, Methods and...

Graphical Methods – Types, Examples and Guide

Symmetric Histogram – Examples and Making Guide

Content Analysis – Methods, Types and Examples

Bimodal Histogram – Definition, Examples

• Search Menu

Sign in through your institution

• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Numismatics
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Papyrology
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Social History
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Acquisition
• Language Evolution
• Language Reference
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Religion
• Music and Media
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Science
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Legal System - Costs and Funding
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Restitution
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Clinical Neuroscience
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Strategy
• Business Ethics
• Business History
• Business and Government
• Business and Technology
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Social Issues in Business and Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic Systems
• Economic History
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Management of Land and Natural Resources (Social Science)
• Natural Disasters (Environment)
• Pollution and Threats to the Environment (Social Science)
• Social Impact of Environmental Issues (Social Science)
• Sustainability
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• Ethnic Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Behaviour
• Political Economy
• Political Institutions
• Political Theory
• Politics and Law
• Politics of Development
• Public Administration
• Public Policy
• Qualitative Political Methodology
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Disability Studies
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

3 Hypothesis Testing in Regression Models

• Published: October 2015
• Cite Icon Cite
• Permissions Icon Permissions

This chapter introduces some key concepts of statistical inference and shows their use to investigate the statistical significance of the (linear) relationships modelled through regression analysis, or to investigate the validity of the classical assumptions in simple and multiple linear regression models. The discussions cover statistical hypothesis testing in simple and multiple regression models; testing linear restrictions on regression coefficients; joint tests of linear restrictions; testing general linear restrictions; the relationship between the F test and the coefficient of multiple correlation; the joint confidence region; multicollinearity and the prediction problem; implications of mis-specification of the regression model on hypothesis testing; Jarque-Bera's test of the normality of regression residuals; the predictive failure test; the Chow test; and non-parametric estimation of the density function. Exercises are provided at the end of the chapter.

Personal account

• Sign in with email/username & password
• Get email alerts
• Save searches
• Purchase content
• Activate your purchase/trial code
• Add your ORCID iD

Institutional access

Sign in with a library card.

• Sign in with username/password
• Recommend to your librarian
• Institutional account management
• Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.

• Click Sign in through your institution.
• Select your institution from the list provided, which will take you to your institution's website to sign in.
• When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
• Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:

• Click Sign in through society site.
• When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

• View your signed in personal account and access account management features.
• View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Our books are available by subscription or purchase to libraries and institutions.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Rights and permissions
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Making Predictions with Regression Analysis

By Jim Frost 37 Comments

If you were able to make predictions about something important to you, you’d probably love that, right? It’s even better if you know that your predictions are sound. In this post, I show how to use regression analysis to make predictions and determine whether they are both unbiased and precise.

You can use regression equations to make predictions. Regression equations are a crucial part of the statistical output after you fit a model. The coefficients in the equation define the relationship between each independent variable and the dependent variable. However, you can also enter values for the independent variables into the equation to predict the mean value of the dependent variable.

Related post : When Should I Use Regression Analysis?

The Regression Approach for Predictions

Using regression to make predictions doesn’t necessarily involve predicting the future. Instead, you predict the mean of the dependent variable given specific values of the independent variable(s). For our example, we’ll use one independent variable to predict the dependent variable. I measured both of these variables at the same point in time.

The general procedure for using regression to make good predictions is the following:

• Research the subject-area so you can build on the work of others. This research helps with the subsequent steps.
• Collect data for the relevant variables.
• Specify and assess your regression model.
• If you have a model that adequately fits the data, use it to make predictions.

While this process involves more work than the psychic approach, it provides valuable benefits. With regression, we can evaluate the bias and precision of our predictions:

• Bias in a statistical model indicates that the predictions are systematically too high or too low.
• Precision represents how close the predictions are to the observed values.

When we use regression to make predictions, our goal is to produce predictions that are both correct on average and close to the real values. In other words, we need predictions that are both unbiased and precise.

Example Scenario for Regression Predictions

We’ll use a regression model to predict body fat percentage based on body mass index (BMI). I collected these data for a study with 92 middle school girls. The variables we measured include height, weight, and body fat measured by a Hologic DXA whole-body system. I’ve calculated the BMI using the height and weight measurements. DXA measurements of body fat percentage are considered to be among the best.

You can download the CSV data file: Predict_BMI .

Why might we want to use BMI to predict body fat percentage? It’s more expensive to obtain your body fat percentage through a direct measure like DXA. If you can use your BMI to predict your body fat percentage, that provides valuable information more easily and cheaply. Let’s see if BMI can produce good predictions!

Finding a Good Regression Model for Predictions

We have the data. Now, we need to determine whether there is a statistically significant relationship between the variables. Relationships, or correlations between variables, are crucial if we want to use the value of one variable to predict the value of another. We also need to evaluate the suitability of the regression model for making predictions.

We have only one independent variable (BMI), so we can use a fitted line plot to display its relationship with body fat percentage. The relationship between the variables is curvilinear. I’ll use a polynomial term to fit the curvature. In this case, I’ll include a quadratic (squared) term. The fitted line plot below suggests that this model fits the data.

Related post : Curve Fitting using Linear and Nonlinear Regression

This curvature is readily apparent because we have only one independent variable and we can graph the relationship. If your model has more than one independent variable, use separate scatterplots to display the association between each independent variable and the dependent variable so you can evaluate the nature of each relationship.

Assess the residual plots

You should also assess the residual plots . If you see patterns in the residual plots, you know that your model is incorrect and that you need to reevaluate it. Non-random residuals indicate that the predicted values are biased. You need to fix the model to produce unbiased predictions.

Learn how to choose the correct regression model .

The residual plots below also confirm the unbiased fit because the data points fall randomly around zero and follow a normal distribution.

Interpret the regression output

In the statistical output below, the p-values indicate that both the linear and squared terms are statistically significant. Based on all of this information, we have a model that provides a statistically significant and unbiased fit to these data. We have a valid regression model. However, there are additional issues we must consider before we can use this model to make predictions.

As an aside, the curved relationship is interesting. The flattening curve indicates that higher BMI values are associated with smaller increases in body fat percentage.

Other Considerations for Valid Predictions

Precision of the predictions.

Previously, we established that our regression model provides unbiased predictions of the observed values. That’s good. However, it doesn’t address the precision of those predictions. Precision measures how close the predictions are to the observed values. We want the predictions to be both unbiased and close to the actual values. Predictions are precise when the observed values cluster close to the predicted values.

Regression predictions are for the mean of the dependent variable. If you think of any mean, you know that there is variation around that mean. The same applies to the predicted mean of the dependent variable. In the fitted line plot, the regression line is nicely in the center of the data points. However, there is a spread of data points around the line. We need to quantify that spread to know how close the predictions are to the observed values. If the spread is too large, the predictions won’t provide useful information.

Later, I’ll generate predictions and show you how to assess the precision.

Related post : Understand Precision in Applied Regression to Avoid Costly Mistakes

Goodness-of-Fit Measures

Goodness-of-fit measures, like R-squared , assess the scatter of the data points around the fitted value. The R-squared for our model is 76.1%, which is good but not great. For a given dataset, higher R-squared values represent predictions that are more precise. However, R-squared doesn’t tell us directly how precise the predictions are in the units of the dependent variable. We can use the standard error of the regression (S) to assess the precision in this manner. However, for this post, I’ll use prediction intervals to evaluate precision.

Related post : Standard Error of the Regression vs. R-squared

New Observations versus Data Used to Fit the Model

R-squared and S indicate how well the model fits the observed data. We need predictions for new observations that the analysis did not use during the model estimation process. Assessing that type of fit requires a different goodness-of-fit measure, the predicted R-squared.

Predicted R-squared measures how well the model predicts the value of new observations. Statistical software packages calculate it by sequentially removing each observation, fitting the model, and determining how well the model predicts the removed observations.

If the predicted R-squared is much lower than the regular R-squared, you know that your regression model doesn’t predict new observations as well as it fits the current dataset. This situation should make you wary of the predictions.

The statistical output below shows that the predicted R-squared (74.14%) is nearly equal to the regular R-squared (76.06%) for our model. We have reason to believe that the model predicts new observations nearly as well as it fits the dataset.

Related post: How to Interpret Adjusted R-squared and Predicted R-squared

Make Predictions Only Within the Range of the Data

Regression predictions are valid only for the range of data used to estimate the model. The relationship between the independent variables and the dependent variable can change outside of that range. In other words, we don’t know whether the shape of the curve changes. If it does, our predictions will be invalid.

The graph shows that the observed BMI values range from 15-35. We should not make predictions outside of this range.

Make Predictions Only for the Population You Sampled

The relationships that a regression model estimates might be valid for only the specific population that you sampled. Our data were collected from middle school girls that are 12-14 years old. The relationship between BMI and body fat percentage might be different for males and different age groups.

Using our Regression Model to Make Predictions

We have a valid regression model that appears to produce unbiased predictions and can predict new observations nearly as well as it predicts the data used to fit the model. Let’s go ahead and use our model to make a prediction and assess the precision.

It is possible to use the regression equation and calculate the predicted values ourselves. However, I’ll use statistical software to do this for us. Not only is this approach easier and more accurate, but I’ll also have it calculate the prediction intervals so we can assess the precision.

I’ll use the software to predict the body fat percentage for a BMI of 18. The prediction output is below.

Interpreting the Regression Prediction Results

The output indicates that the mean value associated with a BMI of 18 is estimated to be ~23% body fat. Again, this mean applies to the population of middle school girls. Let’s assess the precision using the confidence interval (CI) and the prediction interval (PI).

The confidence interval is the range where the mean value for girls with a BMI of 18 is likely to fall. We can be 95% confident that this mean is between 22.1% and 23.9%. However, this confidence interval does not help us evaluate the precision of individual predictions.

A prediction interval is the range where a single new observation is likely to fall. Narrower prediction intervals represent more precise predictions. For an individual middle school girl with a BMI of 18, we can be 95% confident that her body fat percentage is between 16% and 30%.

The range of the prediction interval is always wider than the confidence interval due to the greater uncertainty of predicting an individual value rather than the mean.

Is this prediction sufficiently precise? To make this determination, we’ll need to use our subject-area knowledge in conjunction with any specific requirements we have. I’m not a medical expert, but I’d guess that the 14 point range of 16-30% is too imprecise to provide meaningful information. If this is true, our regression model is too imprecise to be useful.

Don’t Focus On Only the Fitted Values

As we saw in this post, using regression analysis to make predictions is a multi-step process. After collecting the data, you need to specify a valid model. The model must satisfy several conditions before you make predictions. Finally, be sure to assess the precision of the predictions. It’s all too easy to get lulled into a false sense of security by focusing on only the fitted value and not consider the prediction interval.

If you’re learning regression and like the approach I use in my blog, check out my eBook!

Share this:

Reader Interactions

December 30, 2023 at 9:37 pm

Hi Jim, your content is superb – thank you for the valuable resource you provide! I am working on a specific problem. I am attempting to predict the price of a security (actually 2 separate securities) using a number of independent variables, most of which are pricing or other related (forward-looking) securities pricing as well as general economic data. More specifically, I am hoping to predict the price of two separate securities based on the exact same set of independent variables – I am not sure but believe this to be generally known as multivariate multiple linear regression. I have continuous linear values for both dependent and independent variables. However, I’m unclear if there is a specific technique to regress two dependent variables simultaneously on the exact same set of independent variables, or, if I should just regress the dependent variables separately and interpret the results independently (I need both predictions). It should also be noted the dependent variables are inversely correlated with a pearson’s r of -0.610425241 (using about 1,000 datapoints). Also, if there is a good textbook or course you would recommend that covers how to conduct this type of regression, I would greatly appreciate it – I did a decent amount of regression in grad school many years ago but nothing horribly complex. Thank you!

Best, Jim .

May 6, 2023 at 6:56 am

God bless you. This would be perfect, if made in excel, so laymen could have more insight on what is happening. Thank you.

August 4, 2022 at 7:53 am

Thank you for the nice text. I have found in my job that academic research using observational datasets has surprisingly little focus on prediction accuracy of a model. Furthermore, the model selection process is often blurry and the final model might have been chosen quite haphazardly. Some model assumptions might be checked along with some goodness of fit -test, but usually nothing is mentioned about prediction accuracy.

Even the absolute correct model can have large (parameter/function) variance. For prediction, there is also the irreducible error. And even if one uses an unbiased model (unbiased parameter estimates), research shows (Harrell, Zhang, Chatfield, Faraway, Breiman etc.) that model bias will be present. Thus, we don’t even have unbiasedness. And on top of that, we have variance.

I am quite keen on machine learning, where the focus is on prediction accuracy. The approach is kind of like “the proof is in the pudding”. I find it not to be the case for “traditional statistics”, where the aim is more on interpretation (inference). Obviously, a machine learning model is not readily interpretable and it could even be impossible.

If a statistical model focused on inference (interpretation of the parameters) does not predict well, what is its use? If it’s a poor model, it most likely will predict poorly. So you should test that. Even if it’s the correct model, the predictions can be poor because of the variance. Even with an absolute correct unbiased model with large variance, your sample is probably way off from the truth. This leads to poor predictions. How happy can you really be if and when even with a correct model you predict poorly?

Having said all this, I’m leaning towards the opinion that every statistical model should incorporate prediction. Preferably to a new dataset (from the same phenomenon). I think this could help the reproducebility problem disrupting the academic research world.

Any thoughts on this?

October 6, 2021 at 5:49 am

Hello, I enjoy reading through your post. following from South Eastern Kenya University

October 7, 2021 at 11:43 pm

Hi Seku! Welcome to my website! I’m so glad that you’ve found it to be helpful! Happy reading!

September 28, 2021 at 4:14 am

Hello Jim, Thanks a lot for this great post and all sub-links which was really useful for me to understand all the aspects I need to build a regression model and to do forecast. My question is related to multiple regression, what if one important variable is categorical but has many values inside which are difficult to group them. How can I encode it without distorting my model with many numeric category. Thanks a lot

September 29, 2021 at 1:28 am

Coding your categorical variable is a very subject-area specific process. Consequently, I can’t give you a specific answer. However, you’ll need find a system of sorting all your observations into a set of categories. You must find a method so that all observations fall unambiguously into one, and only one, category. These categories must be mutually exclusive. All observations in your study must fall within one category.

November 23, 2020 at 4:27 pm

Hope you are doing well. If a researcher has constructed a new test and would like to investigate to what extent the new test is able to predict the subjects’ performance on an already established test, which test should be taken as a predictor and which one as the outcome measure in the regression analysis?

My intuition is that if the results of the new test can predict subjects’ scores on the old test, we have to consider the new test as the predictor as we are interested in finding out to what extent it can predict the unique variance of the old test.

Thanks in advance and

October 9, 2020 at 4:09 am

hi sir, i have a hypothesis where : amount customers have spent at a store in the last 12 months predicts likelihood they recommend the brand to others. which type of regression would this be and what are the measurements of scale for each IV and DV? thanks!

June 11, 2020 at 11:34 am

Hi Jim, very interesting read. I was wondering, I’ve read a little on Cox for prediction modelling (though not much I’ve found compared to logistic regression models). In prediction time is always important I suppose. Is there any benefit to using Cox over LR? I am looking at risk of developing a condition within 3 years based on certain subject characteristics. Many thanks for your help.

June 7, 2020 at 8:47 am

Hi Jim, an excellent and helpful read thanks. I was hoping you could help me confirm how I would apply the logistic regression equation to generate a risk score for participants to calculate a ROC curve? Thanks!

May 28, 2020 at 9:24 am

Nice explanation. It helped in my project.

May 13, 2020 at 7:08 pm

Hi professor,

I followed up your subjects, really they are valuable and appreciated. However, i have a question, if i have a dependent variable and 4 or 5 independent variables, what is the best method to develop a correct statistical equation which correlate all of them??

May 12, 2020 at 11:18 pm

Hello Sir. How can we predict final exam results from class assignments marks

May 13, 2020 at 3:52 pm

You’re in the right post for the answers you seek! I spell out the process here. If you have more specific questions, please post them after reading thoroughly.

May 9, 2020 at 7:48 am

Hello professor,

Your posts helped me a lot in reshaping my knowledge in regression models. I want to ask you how can we use time as a predictor along side other predictors to perform prediction. What I can’t undrestand is when plotting time against my dependant variable, I find no correlation. So how can I design my study using time?

I hope that I made myself clear.

Thank you again.

May 11, 2020 at 1:11 am

Using regression to analyze time series data is possible. However, it raises a number of other considerations. It’s far too complex to go into in the comments section. However, you should first determine whether time is related to your dependent variable. Instead of a correlation, try graphing it using a time series plot. You can then see if there’s any sort of relationship between time and your DV. Cyclical patterns might not show up as a correlation put would be visible on a time series plot. There’s a bunch of time series analysis methods that you can incorporate into regression analysis. At some point, I might write posts about that. However, it involves many details that I can’t quickly summarize. But, you can factor in the effect of time along with other factors that related to your DV.

I wish I could be more helpful. And perhaps down the road I’ll have something just perfect for you. But alas I don’t right now. I’m sure you could do a search and find more information though.

April 15, 2020 at 12:46 pm

Hello Professor Jim I am a profound admirer of your work and your posts has helped me very much.

When I read this post I thought you were going to mention and talk also about forecasts. But you were talking about regular regressions predictions. So I would like to ask you something important to the scientific investigation I am working on. Do you think that, if besides predict the impact of a IV on a DV, I decide to use the model that I will buld to forecast future values of my dependent variable. Do you think it would add a considerable amount of work? in terms of modelling and code building for the calculations?

Thank you very much.

April 16, 2020 at 11:05 pm

I’m so happy to hear that my posts have been helpful! 🙂

Forecasting in regression uses the same methodology as predictions. You’re still using the IVs to predict the DV. The difference, of course, is that you’re using past values of the IVs to predict future values of the DV. If you’re familiar with fitting regression models, fitting a forecasting model isn’t necessarily going to be more work than a regular regression model. You’ll still need to go through the process of determining which variables to include in your model. Given the forecast nature, you’ll need to think about the variables, the timing of the variables, and how they influence the DV. In addition to the more typical IVs, you’ll need to consider things such as seasonal patterns and other trends over time. Given that the model incorporates time, you will need to pay more attention to the potential problem of autocorrelation in the residuals, which I describe in my post about least squares assumptions . So, there are definitely some different considerations for a forecast model, but, again I wouldn’t say that it’s necessarily harder than a non-forecast model. As usual, it comes down to research, getting the right data, including the correct variables, and checking the assumptions.

I hope this helps!

January 24, 2020 at 1:21 am

I’m starting out in Predictive Analytics and found your article very useful and informative.

I’m currently working on a use case where the quality of a product is directly affected by a temperature parameter (which was found by root cause analysis). So our objective is to maintain the temperature at the nominal value and provide predictions on when the tempertaure may vary. But unfortunately quality data is not available. Hence we need to work with the temperature and additonal process parameters data available to us.

My queries are as follows:

Can I predict the temperature variance and assume that the quality of the product will be in sync to a certain extent ?

Is regression analysis the best methodology for my use case ?

Are there any open source tools available for doing this predictive analytics ?

June 28, 2019 at 9:48 am

Hello dear,

Thank you for all your interesting posts.

I’m beginner in regression and I would like to use logistic Model to predict surrenders in life insurance.

I would like to well understand the prediction probabilities.

In my model I us the age (in months) of the contract in the portefollio, the gender of Policy holder, …

when making prediction, for age 57, gender M for example, what’s does the predicted probability mean?

Does it mean that it’s the probability of the contract to be surrended at age 57 given the gender of the Policy holder?

June 30, 2019 at 8:48 pm

Hi N’Dah,

Yes, the prediction the probability of that a 57 year old male will surrender the policy. That assumes the model provides a good fit and satisfies the necessary assumptions. I write more about binary logistic regression . It’s a post that uses binary logistic regression to analyze a political group in the U.S. But, I do talk about interpreting the output, which might be helpful.

I hope that helps!

May 28, 2019 at 3:26 am

Why is the standard error of estimate or prediction higher when the predictive quality of variables is lower?

May 27, 2019 at 5:41 am

Good Read. Easy to understand keep it up.

May 9, 2019 at 3:31 am

I really appreciate your support in regression analysis. Actually i have data on milk yield of buffaloes. Different buffaloes yield milk in different number of days. in order to rank buffaloes i need to put milk to a standard milk period 305 days. Some buffaloes have lactation length higher than 305 days, other less than 305 days. How to develop factors for correction/prediction of milk of all buffaloes on one standard

May 10, 2019 at 2:09 pm

Hi Musarrat,

The process of identifying the correct variables to include in your model is a mix between subject area knowledge and statistics. To develop an initial list of potential factors, you’ll need to research the subject area and use your expertise to identify candidates. I don’t know the dairy industry so, unfortunately, I can’t help you there.

I suggest you read my post about choosing the correct regression model for some tips. Additionally, consider buying my ebook on regression analysis which can help you with the process.

May 8, 2019 at 2:36 am

Unlike Standard error of regression ( https://statisticsbyjim.com/regression/standard-error-regression-vs-r-squared/ ), the assessment by calculating prediction intervals in this article doesn’t seem to be comprehensive because with SE of regression, it is clear by the rule-of-thumb that a certain number of points must fall within the bounds based on the confidence level (95%, 99%) – this of course depends on how precise we want.

In the case of prediction intervals, the usage of subject matter expertise was mentioned and the calculations were based on every point (where the conditions of independent variables are given). Now, I wonder how to quantify and assess the precision of model based on a one-off calculation?

Considering such scenario, is SE of regression followed/ used typically unless one has a lot of subject expertise and ways to calculate PI for all the data points and subsequently assess the precision of the prediction precision?

Thanks Jim!

October 20, 2018 at 11:36 pm

I am one week before my thesis submission and wish I had found your site much earlier. Your explanations are so clear and concise. You are a great teacher Jim!

October 21, 2018 at 1:29 am

Thank you so much, Keryn! I strive to make these explanations as straightforward as possible, so I really appreciate your kind words!

September 21, 2018 at 10:30 am

Using the body mass index data set as an example. Suppose these results were gained from several different groups. For example one group worked out regularly, one group didn’t work out but maintained a healthy diet, one group didn’t work out and maintained a poor diet, etc. Can we use the group average differences between estimated results (based on the regression equation) and the actual results to determine of one group was significantly different from the others in terms of that group being consistently above or below the regression line?

September 10, 2018 at 12:22 am

Hello, how can I predict the dependent variable for a new case in spss?

April 19, 2018 at 12:36 am

Nice article. Very clear and easy to understand. Bravo.

April 19, 2018 at 2:16 am

Thank you, James!

December 14, 2017 at 2:27 pm

Oh my! I came across you during the final week of my stat class. You just enlightened me in this regression area. I wish I came upon you during my first week of class. It is easier to grasp stats when it is explained plainly and their correlation with whatever in life you will be doing. Safe to say I passed (barely) my stat basically with following step by step without understanding why I am doing it in such a way and why.

December 14, 2017 at 5:00 pm

Hi Ginalyn, thanks for taking the time to write such a nice comment! It made my day! I’m glad you found my blog to be helpful. I always try to explain statistics in the most straightforward, simple manner possible. I’m glad you passed!

September 27, 2017 at 8:46 am

Thanks for the deep insight; indeed your idea brings me back in trying to seek as much closer to reality predictions on our daily life phenomenal. As this universe in as much as the orderly chaotic manner, some predictions becomes erroneously to the extent that they are rendered uncertain for the decision making. In validation of a model in question, the uncertainty would be clarified by using a set of conditions for prediction and suitable intervals (limits).

May 4, 2017 at 11:25 pm

Comments and Questions Cancel reply

• Quality Improvement
• Talk To Minitab

How to Interpret Regression Analysis Results: P-values and Coefficients

Topics: Regression Analysis

Regression analysis generates an equation to describe the statistical relationship between one or more predictor variables and the response variable. After you use Minitab Statistical Software to fit a regression model, and verify the fit by checking the residual plots , you’ll want to interpret the results. In this post, I’ll show you how to interpret the p-values and coefficients that appear in the output for linear regression analysis.

How Do I Interpret the P-Values in Linear Regression Analysis?

The p-value for each term tests the null hypothesis that the coefficient is equal to zero (no effect). A low p-value (< 0.05) indicates that you can reject the null hypothesis. In other words, a predictor that has a low p-value is likely to be a meaningful addition to your model because changes in the predictor's value are related to changes in the response variable.

Conversely, a larger (insignificant) p-value suggests that changes in the predictor are not associated with changes in the response.

In the output below, we can see that the predictor variables of South and North are significant because both of their p-values are 0.000. However, the p-value for East (0.092) is greater than the common alpha level of 0.05, which indicates that it is not statistically significant.

Typically, you use the coefficient p-values to determine which terms to keep in the regression model. In the model above, we should consider removing East.

Related: F-test of overall significance

How Do I Interpret the Regression Coefficients for Linear Relationships?

Regression coefficients represent the mean change in the response variable for one unit of change in the predictor variable while holding other predictors in the model constant. This statistical control that regression provides is important because it isolates the role of one variable from all of the others in the model.

The key to understanding the coefficients is to think of them as slopes, and they’re often called slope coefficients. I’ll illustrate this in the fitted line plot below, where I’ll use a person’s height to model their weight. First, Minitab’s session window output:

The fitted line plot shows the same regression results graphically.

The equation shows that the coefficient for height in meters is 106.5 kilograms. The coefficient indicates that for every additional meter in height you can expect weight to increase by an average of 106.5 kilograms.

The blue fitted line graphically shows the same information. If you move left or right along the x-axis by an amount that represents a one meter change in height, the fitted line rises or falls by 106.5 kilograms. However, these heights are from middle-school aged girls and range from 1.3 m to 1.7 m. The relationship is only valid within this data range, so we would not actually shift up or down the line by a full meter in this case.

If the fitted line was flat (a slope coefficient of zero), the expected value for weight would not change no matter how far up and down the line you go. So, a low p-value suggests that the slope is not zero, which in turn suggests that changes in the predictor variable are associated with changes in the response variable.

I used a fitted line plot because it really brings the math to life. However, fitted line plots can only display the results from simple regression, which is one predictor variable and the response. The concepts hold true for multiple linear regression, but I would need an extra spatial dimension for each additional predictor to plot the results. That's hard to show with today's technology!

How Do I Interpret the Regression Coefficients for Curvilinear Relationships and Interaction Terms?

In the above example, height is a linear effect; the slope is constant, which indicates that the effect is also constant along the entire fitted line. However, if your model requires polynomial or interaction terms, the interpretation is a bit less intuitive.

As a refresher, polynomial terms model curvature in the data , while interaction terms indicate that the effect of one predictor depends on the value of another predictor.

The next example uses a data set that requires a quadratic (squared) term to model the curvature. In the output below, we see that the p-values for both the linear and quadratic terms are significant.

The residual plots (not shown) indicate a good fit, so we can proceed with the interpretation. But, how do we interpret these coefficients? It really helps to graph it in a fitted line plot.

You can see how the relationship between the machine setting and energy consumption varies depending on where you start on the fitted line. For example, if you start at a machine setting of 12 and increase the setting by 1, you’d expect energy consumption to decrease. However, if you start at 25, an increase of 1 should increase energy consumption. And if you’re around 20, energy consumption shouldn’t change much at all.

A significant polynomial term can make the interpretation less intuitive because the effect of changing the predictor varies depending on the value of that predictor. Similarly, a significant interaction term indicates that the effect of the predictor varies depending on the value of a different predictor.

Take extra care when you interpret a regression model that contains these types of terms. You can’t just look at the main effect (linear term) and understand what is happening! Unfortunately, if you are performing multiple regression analysis, you won't be able to use a fitted line plot to graphically interpret the results. This is where subject area knowledge is extra valuable!

Particularly attentive readers may have noticed that I didn’t tell you how to interpret the constant . I’ll cover that in my next post!

Be sure to:

• Check your residual plots so you can trust the results
• Assess the goodness-of-fit and R-squared

If you're learning about regression, read my regression tutorial !

You Might Also Like

• Trust Center

© 2023 Minitab, LLC. All Rights Reserved.

• Terms of Use
• Privacy Policy
• Cookies Settings

Statistics Resources

• Excel - Tutorials
• Basic Probability Rules
• Single Event Probability
• Complement Rule
• Intersections & Unions
• Compound Events
• Levels of Measurement
• Independent and Dependent Variables
• Entering Data
• Central Tendency
• Data and Tests
• Displaying Data
• Discussing Statistics In-text
• SEM and Confidence Intervals
• Two-Way Frequency Tables
• Empirical Rule
• Finding Probability
• Accessing SPSS
• Chart and Graphs
• Frequency Table and Distribution
• Descriptive Statistics
• Converting Raw Scores to Z-Scores
• Converting Z-scores to t-scores
• Split File/Split Output
• Partial Eta Squared
• Downloading and Installing G*Power: Windows/PC
• Correlation
• Testing Parametric Assumptions
• One-Way ANOVA
• Two-Way ANOVA
• Repeated Measures ANOVA
• Goodness-of-Fit
• Test of Association
• Pearson's r
• Point Biserial
• Mediation and Moderation

Simple Linear Regression

Multiple Linear Regression

• Binomial Logistic Regression
• Multinomial Logistic Regression
• Independent Samples T-test
• Dependent Samples T-test
• Testing Assumptions
• T-tests using SPSS
• T-Test Practice
• Predictive Analytics This link opens in a new window
• Quantitative Research Questions
• Null & Alternative Hypotheses
• One-Tail vs. Two-Tail
• Alpha & Beta
• Associated Probability
• Decision Rule
• Statement of Conclusion
• Statistics Group Sessions

Research Questions and Hypotheses

These are just a few examples of what the research questions and hypotheses may look like when a regression analysis is appropriate. 

• H0: Bodyweight does not have an influence on cholesterol levels.
• Ha: Bodyweight has a significant influence on cholesterol levels.
• H0: IQ does not predict GPA.
• Ha: IQ is a significant predictor of GPA.
• H0: Oxygen, water, and sunlight are not related to plant growth.
• Ha: At least one of the predictor variables is a significant predictor of plant growth.
• H0: There is no relationship between IQ or gender, and GPA.
• Ha: IQ and/or gender significantly predict(s) GPA.

Logistic Regression

• H0: Income is not a predictor of gender.
• Ha: There is a predictive relationship between gender and income.
• H0: There is no relationship between customer satisfaction, brand perception, price perception, and purchase decision.
• Ha: At least one of the predictor variables has a predictive relationship with purchase decision.

Multiple Logistic Regression

• H0: There is no influence on game choice by standardized test scores.
• Ha: There is a significant influence of at least one of the predictor variables on game choice.

Was this resource helpful?

• << Previous: Mediation and Moderation
• Next: Simple Linear Regression >>
• Last Updated: Jul 16, 2024 11:19 AM
• URL: https://resources.nu.edu/statsresources

Hypothesis Testing - Analysis of Variance (ANOVA)

Lisa Sullivan, PhD

Professor of Biostatistics

Boston University School of Public Health

Introduction

This module will continue the discussion of hypothesis testing, where a specific statement or hypothesis is generated about a population parameter, and sample statistics are used to assess the likelihood that the hypothesis is true. The hypothesis is based on available information and the investigator's belief about the population parameters. The specific test considered here is called analysis of variance (ANOVA) and is a test of hypothesis that is appropriate to compare means of a continuous variable in two or more independent comparison groups. For example, in some clinical trials there are more than two comparison groups. In a clinical trial to evaluate a new medication for asthma, investigators might compare an experimental medication to a placebo and to a standard treatment (i.e., a medication currently being used). In an observational study such as the Framingham Heart Study, it might be of interest to compare mean blood pressure or mean cholesterol levels in persons who are underweight, normal weight, overweight and obese.  

The technique to test for a difference in more than two independent means is an extension of the two independent samples procedure discussed previously which applies when there are exactly two independent comparison groups. The ANOVA technique applies when there are two or more than two independent groups. The ANOVA procedure is used to compare the means of the comparison groups and is conducted using the same five step approach used in the scenarios discussed in previous sections. Because there are more than two groups, however, the computation of the test statistic is more involved. The test statistic must take into account the sample sizes, sample means and sample standard deviations in each of the comparison groups.

If one is examining the means observed among, say three groups, it might be tempting to perform three separate group to group comparisons, but this approach is incorrect because each of these comparisons fails to take into account the total data, and it increases the likelihood of incorrectly concluding that there are statistically significate differences, since each comparison adds to the probability of a type I error. Analysis of variance avoids these problemss by asking a more global question, i.e., whether there are significant differences among the groups, without addressing differences between any two groups in particular (although there are additional tests that can do this if the analysis of variance indicates that there are differences among the groups).

The fundamental strategy of ANOVA is to systematically examine variability within groups being compared and also examine variability among the groups being compared.

Learning Objectives

After completing this module, the student will be able to:

• Perform analysis of variance by hand
• Appropriately interpret results of analysis of variance tests
• Distinguish between one and two factor analysis of variance tests
• Identify the appropriate hypothesis testing procedure based on type of outcome variable and number of samples

The ANOVA Approach

Consider an example with four independent groups and a continuous outcome measure. The independent groups might be defined by a particular characteristic of the participants such as BMI (e.g., underweight, normal weight, overweight, obese) or by the investigator (e.g., randomizing participants to one of four competing treatments, call them A, B, C and D). Suppose that the outcome is systolic blood pressure, and we wish to test whether there is a statistically significant difference in mean systolic blood pressures among the four groups. The sample data are organized as follows:

The hypotheses of interest in an ANOVA are as follows:

• H 0 : μ 1 = μ 2 = μ 3 ... = μ k
• H 1 : Means are not all equal.

where k = the number of independent comparison groups.

In this example, the hypotheses are:

• H 0 : μ 1 = μ 2 = μ 3 = μ 4
• H 1 : The means are not all equal.

The null hypothesis in ANOVA is always that there is no difference in means. The research or alternative hypothesis is always that the means are not all equal and is usually written in words rather than in mathematical symbols. The research hypothesis captures any difference in means and includes, for example, the situation where all four means are unequal, where one is different from the other three, where two are different, and so on. The alternative hypothesis, as shown above, capture all possible situations other than equality of all means specified in the null hypothesis.

Test Statistic for ANOVA

The test statistic for testing H 0 : μ 1 = μ 2 = ... =   μ k is:

and the critical value is found in a table of probability values for the F distribution with (degrees of freedom) df 1 = k-1, df 2 =N-k. The table can be found in "Other Resources" on the left side of the pages.

NOTE: The test statistic F assumes equal variability in the k populations (i.e., the population variances are equal, or s 1 2 = s 2 2 = ... = s k 2 ). This means that the outcome is equally variable in each of the comparison populations. This assumption is the same as that assumed for appropriate use of the test statistic to test equality of two independent means. It is possible to assess the likelihood that the assumption of equal variances is true and the test can be conducted in most statistical computing packages. If the variability in the k comparison groups is not similar, then alternative techniques must be used.

The F statistic is computed by taking the ratio of what is called the "between treatment" variability to the "residual or error" variability. This is where the name of the procedure originates. In analysis of variance we are testing for a difference in means (H 0 : means are all equal versus H 1 : means are not all equal) by evaluating variability in the data. The numerator captures between treatment variability (i.e., differences among the sample means) and the denominator contains an estimate of the variability in the outcome. The test statistic is a measure that allows us to assess whether the differences among the sample means (numerator) are more than would be expected by chance if the null hypothesis is true. Recall in the two independent sample test, the test statistic was computed by taking the ratio of the difference in sample means (numerator) to the variability in the outcome (estimated by Sp).  

The decision rule for the F test in ANOVA is set up in a similar way to decision rules we established for t tests. The decision rule again depends on the level of significance and the degrees of freedom. The F statistic has two degrees of freedom. These are denoted df 1 and df 2 , and called the numerator and denominator degrees of freedom, respectively. The degrees of freedom are defined as follows:

df 1 = k-1 and df 2 =N-k,

where k is the number of comparison groups and N is the total number of observations in the analysis.   If the null hypothesis is true, the between treatment variation (numerator) will not exceed the residual or error variation (denominator) and the F statistic will small. If the null hypothesis is false, then the F statistic will be large. The rejection region for the F test is always in the upper (right-hand) tail of the distribution as shown below.

Rejection Region for F   Test with a =0.05, df 1 =3 and df 2 =36 (k=4, N=40)

For the scenario depicted here, the decision rule is: Reject H 0 if F > 2.87.

The ANOVA Procedure

We will next illustrate the ANOVA procedure using the five step approach. Because the computation of the test statistic is involved, the computations are often organized in an ANOVA table. The ANOVA table breaks down the components of variation in the data into variation between treatments and error or residual variation. Statistical computing packages also produce ANOVA tables as part of their standard output for ANOVA, and the ANOVA table is set up as follows: 

where  

• X = individual observation,
• k = the number of treatments or independent comparison groups, and
• N = total number of observations or total sample size.

The ANOVA table above is organized as follows.

• The first column is entitled "Source of Variation" and delineates the between treatment and error or residual variation. The total variation is the sum of the between treatment and error variation.
• The second column is entitled "Sums of Squares (SS)" . The between treatment sums of squares is

and is computed by summing the squared differences between each treatment (or group) mean and the overall mean. The squared differences are weighted by the sample sizes per group (n j ). The error sums of squares is:

and is computed by summing the squared differences between each observation and its group mean (i.e., the squared differences between each observation in group 1 and the group 1 mean, the squared differences between each observation in group 2 and the group 2 mean, and so on). The double summation ( SS ) indicates summation of the squared differences within each treatment and then summation of these totals across treatments to produce a single value. (This will be illustrated in the following examples). The total sums of squares is:

and is computed by summing the squared differences between each observation and the overall sample mean. In an ANOVA, data are organized by comparison or treatment groups. If all of the data were pooled into a single sample, SST would reflect the numerator of the sample variance computed on the pooled or total sample. SST does not figure into the F statistic directly. However, SST = SSB + SSE, thus if two sums of squares are known, the third can be computed from the other two.

• The third column contains degrees of freedom . The between treatment degrees of freedom is df 1 = k-1. The error degrees of freedom is df 2 = N - k. The total degrees of freedom is N-1 (and it is also true that (k-1) + (N-k) = N-1).
• The fourth column contains "Mean Squares (MS)" which are computed by dividing sums of squares (SS) by degrees of freedom (df), row by row. Specifically, MSB=SSB/(k-1) and MSE=SSE/(N-k). Dividing SST/(N-1) produces the variance of the total sample. The F statistic is in the rightmost column of the ANOVA table and is computed by taking the ratio of MSB/MSE.  

A clinical trial is run to compare weight loss programs and participants are randomly assigned to one of the comparison programs and are counseled on the details of the assigned program. Participants follow the assigned program for 8 weeks. The outcome of interest is weight loss, defined as the difference in weight measured at the start of the study (baseline) and weight measured at the end of the study (8 weeks), measured in pounds.  

Three popular weight loss programs are considered. The first is a low calorie diet. The second is a low fat diet and the third is a low carbohydrate diet. For comparison purposes, a fourth group is considered as a control group. Participants in the fourth group are told that they are participating in a study of healthy behaviors with weight loss only one component of interest. The control group is included here to assess the placebo effect (i.e., weight loss due to simply participating in the study). A total of twenty patients agree to participate in the study and are randomly assigned to one of the four diet groups. Weights are measured at baseline and patients are counseled on the proper implementation of the assigned diet (with the exception of the control group). After 8 weeks, each patient's weight is again measured and the difference in weights is computed by subtracting the 8 week weight from the baseline weight. Positive differences indicate weight losses and negative differences indicate weight gains. For interpretation purposes, we refer to the differences in weights as weight losses and the observed weight losses are shown below.

Is there a statistically significant difference in the mean weight loss among the four diets?  We will run the ANOVA using the five-step approach.

• Step 1. Set up hypotheses and determine level of significance

H 0 : μ 1 = μ 2 = μ 3 = μ 4 H 1 : Means are not all equal              α=0.05

• Step 2. Select the appropriate test statistic.  

The test statistic is the F statistic for ANOVA, F=MSB/MSE.

• Step 3. Set up decision rule.  

The appropriate critical value can be found in a table of probabilities for the F distribution(see "Other Resources"). In order to determine the critical value of F we need degrees of freedom, df 1 =k-1 and df 2 =N-k. In this example, df 1 =k-1=4-1=3 and df 2 =N-k=20-4=16. The critical value is 3.24 and the decision rule is as follows: Reject H 0 if F > 3.24.

• Step 4. Compute the test statistic.  

To organize our computations we complete the ANOVA table. In order to compute the sums of squares we must first compute the sample means for each group and the overall mean based on the total sample.  

We can now compute

So, in this case:

Next we compute,

SSE requires computing the squared differences between each observation and its group mean. We will compute SSE in parts. For the participants in the low calorie diet:  

For the participants in the low fat diet:  

For the participants in the low carbohydrate diet:  

For the participants in the control group:

We can now construct the ANOVA table .

• Step 5. Conclusion.  

We reject H 0 because 8.43 > 3.24. We have statistically significant evidence at α=0.05 to show that there is a difference in mean weight loss among the four diets.    

ANOVA is a test that provides a global assessment of a statistical difference in more than two independent means. In this example, we find that there is a statistically significant difference in mean weight loss among the four diets considered. In addition to reporting the results of the statistical test of hypothesis (i.e., that there is a statistically significant difference in mean weight losses at α=0.05), investigators should also report the observed sample means to facilitate interpretation of the results. In this example, participants in the low calorie diet lost an average of 6.6 pounds over 8 weeks, as compared to 3.0 and 3.4 pounds in the low fat and low carbohydrate groups, respectively. Participants in the control group lost an average of 1.2 pounds which could be called the placebo effect because these participants were not participating in an active arm of the trial specifically targeted for weight loss. Are the observed weight losses clinically meaningful?

Another ANOVA Example

Calcium is an essential mineral that regulates the heart, is important for blood clotting and for building healthy bones. The National Osteoporosis Foundation recommends a daily calcium intake of 1000-1200 mg/day for adult men and women. While calcium is contained in some foods, most adults do not get enough calcium in their diets and take supplements. Unfortunately some of the supplements have side effects such as gastric distress, making them difficult for some patients to take on a regular basis.  

 A study is designed to test whether there is a difference in mean daily calcium intake in adults with normal bone density, adults with osteopenia (a low bone density which may lead to osteoporosis) and adults with osteoporosis. Adults 60 years of age with normal bone density, osteopenia and osteoporosis are selected at random from hospital records and invited to participate in the study. Each participant's daily calcium intake is measured based on reported food intake and supplements. The data are shown below.   

Is there a statistically significant difference in mean calcium intake in patients with normal bone density as compared to patients with osteopenia and osteoporosis? We will run the ANOVA using the five-step approach.

H 0 : μ 1 = μ 2 = μ 3 H 1 : Means are not all equal                            α=0.05

In order to determine the critical value of F we need degrees of freedom, df 1 =k-1 and df 2 =N-k.   In this example, df 1 =k-1=3-1=2 and df 2 =N-k=18-3=15. The critical value is 3.68 and the decision rule is as follows: Reject H 0 if F > 3.68.

To organize our computations we will complete the ANOVA table. In order to compute the sums of squares we must first compute the sample means for each group and the overall mean.  

 If we pool all N=18 observations, the overall mean is 817.8.

We can now compute:

Substituting:

SSE requires computing the squared differences between each observation and its group mean. We will compute SSE in parts. For the participants with normal bone density:

For participants with osteopenia:

For participants with osteoporosis:

We do not reject H 0 because 1.395 < 3.68. We do not have statistically significant evidence at a =0.05 to show that there is a difference in mean calcium intake in patients with normal bone density as compared to osteopenia and osterporosis. Are the differences in mean calcium intake clinically meaningful? If so, what might account for the lack of statistical significance?

One-Way ANOVA in R

The video below by Mike Marin demonstrates how to perform analysis of variance in R. It also covers some other statistical issues, but the initial part of the video will be useful to you.

Two-Factor ANOVA

The ANOVA tests described above are called one-factor ANOVAs. There is one treatment or grouping factor with k > 2 levels and we wish to compare the means across the different categories of this factor. The factor might represent different diets, different classifications of risk for disease (e.g., osteoporosis), different medical treatments, different age groups, or different racial/ethnic groups. There are situations where it may be of interest to compare means of a continuous outcome across two or more factors. For example, suppose a clinical trial is designed to compare five different treatments for joint pain in patients with osteoarthritis. Investigators might also hypothesize that there are differences in the outcome by sex. This is an example of a two-factor ANOVA where the factors are treatment (with 5 levels) and sex (with 2 levels). In the two-factor ANOVA, investigators can assess whether there are differences in means due to the treatment, by sex or whether there is a difference in outcomes by the combination or interaction of treatment and sex. Higher order ANOVAs are conducted in the same way as one-factor ANOVAs presented here and the computations are again organized in ANOVA tables with more rows to distinguish the different sources of variation (e.g., between treatments, between men and women). The following example illustrates the approach.

Consider the clinical trial outlined above in which three competing treatments for joint pain are compared in terms of their mean time to pain relief in patients with osteoarthritis. Because investigators hypothesize that there may be a difference in time to pain relief in men versus women, they randomly assign 15 participating men to one of the three competing treatments and randomly assign 15 participating women to one of the three competing treatments (i.e., stratified randomization). Participating men and women do not know to which treatment they are assigned. They are instructed to take the assigned medication when they experience joint pain and to record the time, in minutes, until the pain subsides. The data (times to pain relief) are shown below and are organized by the assigned treatment and sex of the participant.

Table of Time to Pain Relief by Treatment and Sex

The analysis in two-factor ANOVA is similar to that illustrated above for one-factor ANOVA. The computations are again organized in an ANOVA table, but the total variation is partitioned into that due to the main effect of treatment, the main effect of sex and the interaction effect. The results of the analysis are shown below (and were generated with a statistical computing package - here we focus on interpretation). 

 ANOVA Table for Two-Factor ANOVA

There are 4 statistical tests in the ANOVA table above. The first test is an overall test to assess whether there is a difference among the 6 cell means (cells are defined by treatment and sex). The F statistic is 20.7 and is highly statistically significant with p=0.0001. When the overall test is significant, focus then turns to the factors that may be driving the significance (in this example, treatment, sex or the interaction between the two). The next three statistical tests assess the significance of the main effect of treatment, the main effect of sex and the interaction effect. In this example, there is a highly significant main effect of treatment (p=0.0001) and a highly significant main effect of sex (p=0.0001). The interaction between the two does not reach statistical significance (p=0.91). The table below contains the mean times to pain relief in each of the treatments for men and women (Note that each sample mean is computed on the 5 observations measured under that experimental condition).  

Mean Time to Pain Relief by Treatment and Gender

Treatment A appears to be the most efficacious treatment for both men and women. The mean times to relief are lower in Treatment A for both men and women and highest in Treatment C for both men and women. Across all treatments, women report longer times to pain relief (See below).  

Notice that there is the same pattern of time to pain relief across treatments in both men and women (treatment effect). There is also a sex effect - specifically, time to pain relief is longer in women in every treatment.  

Suppose that the same clinical trial is replicated in a second clinical site and the following data are observed.

Table - Time to Pain Relief by Treatment and Sex - Clinical Site 2

The ANOVA table for the data measured in clinical site 2 is shown below.

Table - Summary of Two-Factor ANOVA - Clinical Site 2

Notice that the overall test is significant (F=19.4, p=0.0001), there is a significant treatment effect, sex effect and a highly significant interaction effect. The table below contains the mean times to relief in each of the treatments for men and women.  

Table - Mean Time to Pain Relief by Treatment and Gender - Clinical Site 2

Notice that now the differences in mean time to pain relief among the treatments depend on sex. Among men, the mean time to pain relief is highest in Treatment A and lowest in Treatment C. Among women, the reverse is true. This is an interaction effect (see below).  

Notice above that the treatment effect varies depending on sex. Thus, we cannot summarize an overall treatment effect (in men, treatment C is best, in women, treatment A is best).    

When interaction effects are present, some investigators do not examine main effects (i.e., do not test for treatment effect because the effect of treatment depends on sex). This issue is complex and is discussed in more detail in a later module. 

Hypothesis Test for Regression and Correlation Analysis

Do you still remember the term hypothesis? The hypothesis is a temporary conclusion from the research results that will conduct. You can also state that the hypothesis is a preliminary answer from the research results based on our presumptions. In this article, I will discuss the Hypothesis Test for Regression and Correlation Analysis on this occasion.

Determining the hypothesis must justify based on theoretical references and empirical study. This justification can be arranged by reading more reference books related to the research topic. Suppose in economic theory; there is a law of demand. When the price increases, the quantity of demand will decrease; vice versa, demand will increase when the price decreases.

Then in formulating the hypothesis, it can be based on empirical study. The empirical study can be obtained from previous related studies that have been tested empirically and have been tested using statistics. Regression and correlation are very popular in the world of research, yes. If you open Google Scholar, you will find many research publications that use regression and correlation analysis.

“Why do many researchers choose regression and correlation as to their analytical tools?” the answer is quite simple, “Because it fits the purpose of our research!” So, regression analysis is used to analyze the effect of one variable on other variables. Then correlation analysis is used to analyze the relationship between variables. Both regression and correlation, some assumptions must be passed. In testing the hypothesis for regression and correlation can used two ways, namely:

Hypothesis Test with P-Value

In testing the hypothesis, it can be seen from the p-value. In formulating a hypothesis, we need to determine an alpha value that we will use. Experimental studies generally use 5% and 1%. In socio-economic, the alpha value limit between 5% and 10% with justification for the research environment is not fully controllable. The meaning of the p-value of alpha 5% is that when the experiment is carried out 100 times and the failure is five times; the research has succeeded. We can also say that the confidence level is greater than 95%.

Suppose there is a study that aims to find out how the effect of price on sales. In this study, alpha was set at 5%. You can formulate research hypotheses in the null hypothesis and alternative hypothesis as follows:

Ho = Price has no significant effect on sales

Ha = price has a significant effect on sales

Next, we can test using simple linear regression. Based on the analysis results, you will get the calculated F value, T count, and p-value (sig.)

Hypothesis testing criteria can follow these rules:

1. p-value (sig.) > 0.05, the null hypothesis is accepted

2. p-value (sig.) ≤ 0.05, the null hypothesis is rejected (accepted alternative hypothesis).

If the regression analysis results, for example, the p-value of the T-test is less than 0.05, the null hypothesis (Ho) is rejected (accepting the alternative hypothesis). Thus, it can be concluded that the price has a significant effect on sales.

Hypothesis Testing by comparing Statistical Tables

Then you can use alternative hypothesis testing by comparing the t value with the t table. Using the p-value criterion alone is sufficient, but it is also important to know the alternative criteria. This criterion is very important if you do manual regression analysis calculations using a calculator.

Using this alternative is the same as the first method. You need to compare the t value with the t table. Suppose we use the same case study example, you can formulate research hypotheses in the null hypothesis and alternative hypothesis as follows:

Ho = Price has no significant effect on sales Ha = price has a significant effect on sales

Next, for hypothesis testing, you can follow these rules:

1. T-value < T table then the null hypothesis is accepted

2. T-value ≥ T table means the null hypothesis is rejected (accepting alternative hypothesis).

If the T value is greater than T-table, the null hypothesis (Ho) is rejected, or the alternative hypothesis (Ha) is accepted. In conclusion, the price has a significant effect on sales. Well, I hope this article is useful for you. See you in the next article. Thank you, bye.

Difference Among Regression, Correlation, and Comparative Test

Why is Descriptive Statistical Analysis Important?

Leave a comment cancel reply.

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Pardon Our Interruption

As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:

• You've disabled JavaScript in your web browser.
• You're a power user moving through this website with super-human speed.
• You've disabled cookies in your web browser.
• A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this support article .

To regain access, please make sure that cookies and JavaScript are enabled before reloading the page.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Neuromarketing and big data analysis of banking firms’ website interfaces and performance.

1. Introduction

2. literature background, 2.1. banking firms, digital marketing, and user engagement, 2.2. metrics and kpis of friendly website user interface (ui), 2.3. neuromarketing and big data analysis implications on website interface and performance, 2.4. hypotheses development, 3. materials and methods, 3.1. methodological concept.

• The research started with the collection of data on website customers and digital marketing activities from banking firm websites. A website’s user behavioral data (pages per visit, bounce rate, time on site, etc.) were sourced from the website platform Semrush [ 61 ], which enables the extraction of big data from corporate webpages.
• The next step involved statistical analysis using methods such as descriptive statistics, correlation, and linear regression. By analyzing the coefficients obtained, researchers can determine the impact of banking firms’ website customer data on their digital marketing and interface performance metrics, including purchase conversion, display ads, organic traffic, and bounce rate.
• After statistical analysis, a hybrid model (HM) incorporating agent-based models (ABMs) and System Dynamics (SD) was used for the simulation. The software AnyLogic (version 8.9.1) [ 62 ] was employed to create a hybrid model that simulates the relationships between the study’s dependent and independent variables over 360 days. This model aims to represent the dynamic interaction between banking firms’ website interface metrics and key metrics of their digital marketing strategies.
• The final stage included a neuromarketing approach to gain deeper insights from 26 participants who viewed the websites of the selected banking firms. They were instructed to search and observe, in 20 s, the selected banking firm websites and their provided financial products and services. Eye-tracking and heatmap analysis were conducted using the SeeSo Web Analysis platform (Eyedid SDK) [ 63 ]. This method seeks to extract additional information about the onsite activity and engagement of the participants from the qualitative methodological concept.

3.2. Fuzzy Cognitive Mapping (FCM) Framework

3.3. research sample, 4.1. statistical analysis, 4.2. simulation model, 4.3. neuromarketing applications, 5. discussion, 6. conclusions, 6.1. theoretical, practical, and managerial implications, 6.2. future work and limitations, author contributions, data availability statement, conflicts of interest.

• Hennig-Thurau, T.; Malthouse, E.C.; Friege, C.; Gensler, S.; Lobschat, L.; Rangaswamy, A.; Skiera, B. The impact of new media on customer relationships. J. Serv. Res. 2010 , 13 , 311–330. [ Google Scholar ] [ CrossRef ]
• Broby, D. Financial technology and the future of banking. Financ. Innov. 2021 , 7 , 1–19. [ Google Scholar ] [ CrossRef ]
• Ding, Q.; He, W. Digital transformation, monetary policy and risk-taking of banks. Financ. Res. Lett. 2023 , 55 , 103986. [ Google Scholar ] [ CrossRef ]
• Shukla, S. Analyzing customer engagement through e-CRM: The role of relationship marketing in the era of digital banking in Varanasi banks. J. Commer. Econ. Comput. Sci. 2021 , 7 , 57–65. [ Google Scholar ]
• Hendriyani, C.; Raharja, S.J. Analysis building customer engagement through eCRM in the era of digital banking in Indonesia. Int. J. Econ. Policy Emerg. Econ. 2018 , 11 , 479–486. [ Google Scholar ]
• Vivek, S.D.; Beatty, S.E.; Morgan, R.M. Customer engagement: Exploring customer relationships beyond purchase. J. Mark. Theory Pract. 2012 , 20 , 122–146. [ Google Scholar ] [ CrossRef ]
• Lee, D.; Hosanagar, K.; Nair, H.S. Advertising content and consumer engagement on social media: Evidence from Facebook. Manag. Sci. 2018 , 64 , 5105–5131. [ Google Scholar ] [ CrossRef ]
• Lin, K.-Y.; Lu, H.-P. Why people use social networking sites: An empirical study integrating network externalities and motivation theory. Comput. Hum. Behav. 2011 , 27 , 1152–1161. [ Google Scholar ] [ CrossRef ]
• Lee, M.; Wang, Y.R.; Huang, C.F. Design and development of a friendly user interface for building construction traceability system. Microsyst. Technol. 2021 , 27 , 1773–1785. [ Google Scholar ] [ CrossRef ]
• Faghih, B.; Azadehfar, M.; Katebi, S. User interface design for E-learning software. Int. J. Soft Comput. Softw. Eng. 2014 , 3 , 786–794. [ Google Scholar ] [ CrossRef ]
• Cheng, S.; Yang, Y.; Xiu, L.; Yu, G. Effects of prior experience on the user experience of news aggregation app’s features—Evidence from a behavioral experiment. Int. J. Hum.-Comput. Interact. 2022 , 39 , 1271–1279. [ Google Scholar ] [ CrossRef ]
• Nielsen, J.; Norman, D. The Definition of User Experience (UX) ; Nielsen Norman Group N N/g.: Fremont, CA, USA, 2018; Available online: https://www.nngroup.com/articles/definition-user-experience/ (accessed on 20 June 2024).
• He, W.; Hung, J.-L.; Liu, L. Impact of big data analytics on banking: A case study. J. Enterp. Inf. Manag. 2023 , 36 , 459–479. [ Google Scholar ] [ CrossRef ]
• Kalaganis, F.P.; Georgiadis, K.; Oikonomou, V.P.; Laskaris, N.A.; Nikolopoulos, S.; Kompatsiaris, I. Unlocking the Subconscious Consumer Bias: A Survey on the Past, Present, and Future of Hybrid EEG Schemes in Neuromarketing. Front. Neuroergonomics 2021 , 2 , 672982. [ Google Scholar ] [ CrossRef ]
• Walker, P.R. How Does Website Design in the e-Banking Sector Affect Customer Attitudes and Behaviour? Ph.D. Thesis, University of Northumbria, Newcastle upon Tyne, UK, 2021. Available online: https://nrl.northumbria.ac.uk/id/eprint/5849/7/walker.philip_phd_(VOLUME_1of2).pdf (accessed on 12 June 2024).
• Manser Payne, E.H.; Peltier, J.; Barger, V.A. Enhancing the value co-creation process: Artificial intelligence and mobile banking service platforms. J. Res. Interact. Mark. 2021 , 15 , 68–85. [ Google Scholar ] [ CrossRef ]
• Diener, F.; Špacek, M. Digital transformation in banking: A managerial perspective on barriers to change. Sustainability 2021 , 13 , 2032. [ Google Scholar ] [ CrossRef ]
• Khattak, M.A.; Ali, M.; Azmi, W.; Rizvi, S.A.R. Digital transformation, diversification and stability: What do we know about banks? Econ. Anal. Policy 2023 , 78 , 122–132. [ Google Scholar ] [ CrossRef ]
• Giannakis-Bompolis, C.; Boutsouki, C. Customer Relationship Management in the Era of Social Web and Social Customer: An Investigation of Customer Engagement in the Greek Retail Banking Sector. Procedia Soc. Behav. Sci. 2014 , 148 , 67–78. [ Google Scholar ] [ CrossRef ]
• Mogaji, E. Redefining banks in the digital era: A typology of banks and their research, managerial and policy implications. Int. J. Bank Mark. 2023 , 41 , 1899–1918. [ Google Scholar ] [ CrossRef ]
• Salvi, A.; Petruzzella, F.; Raimo, N.; Vitolla, F. Transparency in the digitalization choices and the cost of equity capital. Qual. Res. Financ. Mark. 2023 , 15 , 630–646. [ Google Scholar ] [ CrossRef ]
• Carmona, J.; Cruz, C. Banks’ social media goals and strategies. J. Bus. Res. 2018 , 91 , 31–41. [ Google Scholar ] [ CrossRef ]
• Kosiba, J.P.; Boateng, H.; Okoe, A.F.; Hinson, R. Trust and customer engagement in the banking sector in Ghana. Serv. Ind. J. 2018 , 40 , 960–973. [ Google Scholar ] [ CrossRef ]
• Del Sarto, N.; Bocchialini, E.; Gai, L.; Ielasi, F. Digital banking: How social media is shaping the game. Qual. Res. Financ. Mark. 2024 . ahead of print . [ Google Scholar ] [ CrossRef ]
• Sakas, D.P.; Giannakopoulos, N.T.; Trivellas, P. Exploring affiliate marketing’s impact on customers’ brand engagement and vulnerability in the online banking service sector. Int. J. Bank Mark. 2023 , 42 , 1282–1312. [ Google Scholar ] [ CrossRef ]
• Sakas, D.P.; Giannakopoulos, N.T.; Terzi, M.C.; Kamperos, I.D.G.; Kanellos, N. What is the connection between Fintechs’ video marketing and their vulnerable customers’ brand engagement during crises? Int. J. Bank Mark. 2023 , 42 , 1313–1347. [ Google Scholar ] [ CrossRef ]
• Mbama, C.I.; Ezepue, P.O. Digital banking, customer experience and bank financial performance: UK customers’ perceptions. Int. J. Bank Mark. 2018 , 36 , 230–255. [ Google Scholar ] [ CrossRef ]
• Khandelwal, R.; Kapoor, D. The Use of Digital Tools for Customer Engagement in the Financial Services Sector. In Revolutionizing Customer-Centric Banking through ICT ; IGI Global: Hershey, PA, USA, 2024; pp. 29–55. [ Google Scholar ]
• Islam, J.U.; Shahid, S.; Rasool, A.; Rahman, Z.; Khan, I.; Rather, R.A. Impact of website attributes on customer engagement in banking: A solicitation of stimulus-organism-response theory. Int. J. Bank Mark. 2020 , 38 , 1279–1303. [ Google Scholar ] [ CrossRef ]
• Lestari, D.M.; Hardianto, D.; Hidayanto, A.N. Analysis of user experience quality on responsive web design from its informative perspective. Int. J. Softw. Eng. Appl. 2014 , 8 , 53–62. [ Google Scholar ] [ CrossRef ]
• Almeida, F.; Monteiro, J. Approaches and principles for UX web experiences: A case study approach. Int. J. Inf. Technol. Web Eng. 2017 , 12 , 49–65. [ Google Scholar ] [ CrossRef ]
• Walsh, T.A.; Kapfhammer, G.M.; McMinn, P. Automated layout failure detection for responsive web pages without an explicit oracle. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, Santa Barbara, CA, USA, 10–14 July 2017. [ Google Scholar ] [ CrossRef ]
• Rogers, Y.; Sharp, H.; Preece, J. Interaction Design: Beyond Human-Computer Interaction , 6th ed.; John Wiley & Sons Ltd.: New York, NY, USA, 2023. [ Google Scholar ]
• ISO9241-11 ; Ergonomics of Human-System Interaction–Part 11: Usability for Definition and Concept. ISO: Geneva, Switzerland, 2018.
• Hussain, I.; Khan, I.A.; Jadoon, W.; Jadoon, R.N.; Khan, A.N.; Shafi, M. Touch or click friendly: Towards adaptive user interfaces for complex applications. PLoS ONE 2024 , 19 , e0297056. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Kim, S.; Cho, D. Technology Trends for UX/UI of Smart Contents. Korea Contents Assoc. Rev. 2016 , 14 , 29–33. [ Google Scholar ] [ CrossRef ]
• Joo, H.S. A Study on UI/UX and Understanding of Computer Major Students. Int. J. Adv. Smart Converg. 2017 , 6 , 26–32. [ Google Scholar ]
• Von Saucken, C.; Michailidou, I.; Lindemann, U. How to Design Experiences: Macro UX versus Micro UX Approach. Lect. Notes Comuter Sci. 2013 , 8015 , 130–139. [ Google Scholar ]
• Instatus. Our Comprehensive List of Website Performance Metrics to Monitor. 2024. Available online: https://instatus.com/blog/website-performance-metrics (accessed on 20 June 2024).
• Levrini, G.R.; Jeffman dos Santos, M. The influence of Price on purchase intentions: Comparative study between cognitive, sensory, and neurophysiological experiments. Behav. Sci. 2021 , 11 , 16. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Gabriel, D.; Merat, E.; Jeudy, A.; Cambos, S.; Chabin, T.; Giustiniani, J.; Haffen, E. Emotional effects induced by the application of a cosmetic product: A real-time electrophysiological evaluation. Appl. Sci. 2021 , 11 , 4766. [ Google Scholar ] [ CrossRef ]
• Filipović, F.; Baljak, L.; Naumović, T.; Labus, A.; Bogdanović, Z. Developing a web application for recognizing emotions in neuromarketing. In Marketing and Smart Technologies ; Springer: Berlin/Heidelberg, Germany, 2020; pp. 297–308. [ Google Scholar ]
• Lee, Ν.; Broderick, A.J.; Chamberlain, L. What is ‘neuromarketing’? A discussion and agenda for future research. Int. J. Psychophysiol. 2007 , 63 , 199–204. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Rawnaque, F.; Rahman, K.; Anwar, S.; Vaidyanathan, R.; Chau, T.; Sarker, F.; Mamun, K. Technological advancements and opportunities in Neuromarketing: A systematic review. Brain Inform. 2020 , 7 , 10. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Ariely, D.; Berns, G. Neuromarketing: The hope and hype of neuroimaging in business. Nat. Rev. Neurosci. 2010 , 11 , 284–292. [ Google Scholar ] [ CrossRef ]
• Sousa, J. Neuromarketing and Big Data Analytics for Strategic Consumer Engagement: Emerging Research and Opportunities ; IGI Global: Hershey, PA, USA, 2017. [ Google Scholar ] [ CrossRef ]
• Šola, H.M.; Qureshi, F.H.; Khawaja, S. Exploring the Untapped Potential of Neuromarketing in Online Learning: Implications and Challenges for the Higher Education Sector in Europe. Behav. Sci. 2024 , 14 , 80. [ Google Scholar ] [ CrossRef ]
• Berčík, J.; Neomániová, K.; Gálová, J. Using neuromarketing to understand user experience with the website (UX) and interface (UI) of a selected company. In The Poprad Economic and Management Forum 2021, Conference Proceedings from International Scientific Conference, Poprad, Slovak Republic, 14 October 2021 ; Madzík, P., Janošková, M., Eds.; VERBUM: Ružomberok, Slovakia, 2021; pp. 246–254. [ Google Scholar ]
• Golnar-Nik, P.; Farashi, S.; Safari, M. The application of EEG power for the prediction and interpretation of consumer decision-making: A neuromarketing study. Physiol. Behav. 2019 , 207 , 90–98. [ Google Scholar ] [ CrossRef ]
• Uygun, Y.; Oguz, R.F.; Olmezogullari, E.; Aktas, M.S. On the Large-scale Graph Data Processing for User Interface Testing in Big Data Science Projects. In Proceedings of the 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 10–13 December 2020; pp. 2049–2056. [ Google Scholar ] [ CrossRef ]
• Li, L.; Zhang, J. Research and Analysis of an Enterprise E-Commerce Marketing System under the Big Data Environment. J. Organ. End User Comput. 2021 , 33 , 1–19. [ Google Scholar ] [ CrossRef ]
• Sakas, D.P.; Giannakopoulos, N.T.; Terzi, M.C.; Kanellos, N.; Liontakis, A. Digital Transformation Management of Supply Chain Firms Based on Big Data from DeFi Social Media Profiles. Electronics 2023 , 12 , 4219. [ Google Scholar ] [ CrossRef ]
• Bala, M.; Verma, D. A Critical Review of Digital Marketing. Int. J. Manag. IT Eng. 2018 , 8 , 321–339. Available online: https://ssrn.com/abstract=3545505 (accessed on 20 July 2024).
• Pongpaew, W.; Speece, M.; Tiangsoongnern, L. Social presence and customer brand engagement on Facebook brand pages. J. Prod. Brand Manag. 2017 , 26 , 262–281. [ Google Scholar ] [ CrossRef ]
• Chaffey, D.; Ellis-Chadwick, F. Digital Marketing ; Pearson: London, UK, 2019. [ Google Scholar ]
• Dodson, I. The Art of Digital Marketing: The Definitive Guide to Creating Strategic, Targeted, and Measurable Online Campaigns ; John Wiley & Sons: New York, NY, USA, 2016. [ Google Scholar ]
• Chawla, Y.; Chodak, G. Social media marketing for businesses: Organic promotions of web-links on Facebook. J. Bus. Res. 2021 , 135 , 49–65. [ Google Scholar ] [ CrossRef ]
• McIlwain, C.D. Algorithmic Discrimination: A Framework and Approach to Auditing & Measuring the Impact of Race-Targeted Digital Advertising. PolicyLink Rep. 2023 , 1–50. [ Google Scholar ] [ CrossRef ]
• Mladenović, D.; Rajapakse, A.; Kožuljević, N.; Shukla, Y. Search engine optimization (SEO) for digital marketers: Exploring determinants of online search visibility for blood bank service. Online Inf. Rev. 2023 , 47 , 661–679. [ Google Scholar ] [ CrossRef ]
• Wedel, M.; Kannan, P.K. Marketing analytics for data-rich environments. J. Mark. 2016 , 80 , 97–121. [ Google Scholar ] [ CrossRef ]
• Semrush. 2024. Available online: https://www.semrush.com/ (accessed on 12 April 2024).
• Anylogic. 2024. Available online: https://www.anylogic.com/ (accessed on 12 April 2024).
• SeeSo Web Analysis (Eyedid SDK). 2024. Available online: https://sdk.eyedid.ai/ (accessed on 20 April 2024).
• MentalModeler. 2024. Available online: https://dev.mentalmodeler.com/ (accessed on 10 April 2024).
• Migkos, S.P.; Sakas, D.P.; Giannakopoulos, N.T.; Konteos, G.; Metsiou, A. Analyzing Greece 2010 Memorandum’s Impact on Macroeconomic and Financial Figures through FCM. Economies 2022 , 10 , 178. [ Google Scholar ] [ CrossRef ]
• Mpelogianni, V.; Groumpos, P.P. Re-approaching fuzzy cognitive maps to increase the knowledge of a system. AI Soc. 2018 , 33 , 175–188. [ Google Scholar ] [ CrossRef ]
• Forbes India. The 10 Largest Banks in the World in 2024. 2024. Available online: https://www.forbesindia.com/article/explainers/the-10-largest-banks-in-the-world/86967/1 (accessed on 6 January 2024).
• Nugroho, S.; Uehara, T. Systematic Review of Agent-Based and System Dynamics Models for Social-Ecological System Case Studies. Systems 2023 , 11 , 530. [ Google Scholar ] [ CrossRef ]
• McGarraghy, S.; Olafsdottir, G.; Kazakov, R.; Huber, É.; Loveluck, W.; Gudbrandsdottir, I.Y.; Čechura, L.; Esposito, G.; Samoggia, A.; Aubert, P.-M.; et al. Conceptual System Dynamics and Agent-Based Modelling Simulation of Interorganisational Fairness in Food Value Chains: Research Agenda and Case Studies. Agriculture 2022 , 12 , 280. [ Google Scholar ] [ CrossRef ]
• Wang, H.; Shi, W.; He, W.; Xue, H.; Zeng, W. Simulation of urban transport carbon dioxide emission reduction environment economic policy in China: An integrated approach using agent-based modelling and system dynamics. J. Clean. Prod. 2023 , 392 , 136221. [ Google Scholar ] [ CrossRef ]
• Nguyen, L.K.N.; Howick, S.; Megiddo, I. A framework for conceptualising hybrid system dynamics and agent-based simulation model. Eur. J. Oper. Res. 2024 , 315 , 1153–1166. [ Google Scholar ] [ CrossRef ]
• Ezquerra, A.; Agen, F.; Bogdan Toma, R.; Ezquerra-Romano, I. Using facial emotion recognition to research emotional phases in an inquiry-based science activity. Res. Sci. Technol. Educ. 2023 , 1–24. [ Google Scholar ] [ CrossRef ]
• Chen, Y.; Qin, X.; Xu, X. Visual Analysis and Recognition of Virtual Reality Resolution Based on Pupil Response and Galvanic Skin Response. In Proceedings of the 4th International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI) 2023, Guangzhou, China, 4–6 August 2023; pp. 74–83. [ Google Scholar ] [ CrossRef ]
• Muke, P.Z.; Kozierkiewicz, A.; Pietranik, M. Investigation and Prediction of Cognitive Load During Memory and Arithmetic Tasks. In Computational Collective Intelligence. ICCCI 2023. Lecture Notes in Computer Science ; Nguyen, N.T., Botzheim, J., Gulyás, L., Núñez, M., Treur, J., Vossen, G., Kozierkiewicz, A., Eds.; Springer: Cham, Switzerland, 2023; Volume 14162. [ Google Scholar ] [ CrossRef ]
• Amiri, S.S.; Masoudi, M.; Asadi, S.; Karan, E.P. A Quantitative Way for Measuring the Building User Design Feedback and Evaluation. In Proceedings of the 16th International Conference on Computing in Civil and Building Engineering (ICCCBE2016), Osaka, Japan, 6–8 July 2016; pp. 1–7. [ Google Scholar ]
• Wilson, L. 30-Minute Conversion Rate Optimisation Actions. In 30-Minute Website Marketing ; Emerald Publishing Limited: Leeds, UK, 2019; pp. 131–141. [ Google Scholar ] [ CrossRef ]
• Sood, S. Leveraging Web Analytics for Optimizing Digital Marketing Strategies. In Big Data Analytics ; Chaudhary, K., Alam, M., Eds.; CRC Press (Auerbach Publications): Boca Raton, FL, USA, 2022; pp. 173–188. [ Google Scholar ]
• Drivas, I.C.; Sakas, D.P.; Giannakopoulos, G.A. Display Advertising and Brand Awareness in Search Engines: Predicting the Engagement of Branded Search Traffic Visitors. In Business Intelligence and Modelling. IC-BIM 2019. Springer Proceedings in Business and Economics ; Sakas, D.P., Nasiopoulos, D.K., Taratuhina, Y., Eds.; Springer: Cham, Switzerland, 2021. [ Google Scholar ] [ CrossRef ]
• Hari, H.; Iyer, R.; Sampat, B. Customer Brand Engagement through Chatbots on Bank Websites–Examining the Antecedents and Consequences. Int. J. Hum. Comput. Interact. 2023 , 38 , 1212–1227. [ Google Scholar ] [ CrossRef ]
• Makrydakis, N. SEO mix 6 O’s model and categorization of search engine marketing factors for websites ranking on search engine result pages. Int. J. Res. Mark. Manag. Sales 2024 , 6 , 18–32. [ Google Scholar ] [ CrossRef ]
• Shankar, B. Strategies for Deep Customer Engagement. In Nuanced Account Management ; Palgrave Macmillan: Singapore, 2018; pp. 53–99. [ Google Scholar ] [ CrossRef ]
• Chakrabortty, K.; Jose, E. Relationship Analysis between Website Traffic, Domain Age and Google Indexed Pages of E-commerce Websites. IIM Kozhikode Soc. Manag. Rev. 2018 , 7 , 171–177. [ Google Scholar ] [ CrossRef ]
• Müller, O.; Fay, M.; vom Brocke, J. The Effect of Big Data and Analytics on Firm Performance: An Econometric Analysis Considering Industry Characteristics. J. Manag. Inf. Syst. 2018 , 35 , 488–509. [ Google Scholar ] [ CrossRef ]
• Pejić Bach, M.; Krstić, Ž.; Seljan, S.; Turulja, L. Text Mining for Big Data Analysis in Financial Sector: A Literature Review. Sustainability 2019 , 11 , 1277. [ Google Scholar ] [ CrossRef ]
• Gupta, S.; Justy, T.; Kamboj, S.; Kumar, A.; Kristoffersen, E. Big data and firm marketing performance: Findings from knowledge-based view. Technol. Forecast. Soc. Change 2021 , 171 , 120986. [ Google Scholar ] [ CrossRef ]
• Ravi, V.; Kamaruddin, S. Big Data Analytics Enabled Smart Financial Services: Opportunities and Challenges. In Big Data Analytics. BDA 2017. Lecture Notes in Computer Science ; Reddy, P., Sureka, A., Chakravarthy, S., Bhalla, S., Eds.; Springer: Cham, Switzerland, 2017; Volume 10721, pp. 15–39. [ Google Scholar ] [ CrossRef ]
• Tichindelean, M.T.; Cetină, I.; Orzan, G. A Comparative Eye Tracking Study of Usability—Towards Sustainable Web Design. Sustainability 2021 , 13 , 10415. [ Google Scholar ] [ CrossRef ]
• Bajaj, R.; Syed, A.A.; Singh, S. Analysing applications of neuromarketing in efficacy of programmatic advertising. J. Consum. Behav. 2023 , 23 , 939–958. [ Google Scholar ] [ CrossRef ]
• Tirandazi, P.; Bamakan, S.M.H.; Toghroljerdi, A. A review of studies on internet of everything as an enabler of neuromarketing methods and techniques. J. Supercomput. 2022 , 79 , 7835–7876. [ Google Scholar ] [ CrossRef ]
• Slijepčević, M.; Popović Šević, N.; Radojević, I.; Šević, A. Relative Importance of Neuromarketing in Support of Banking Service Users. Marketing 2022 , 53 , 131–142. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Giannakopoulos, N.T.; Sakas, D.P.; Migkos, S.P. Neuromarketing and Big Data Analysis of Banking Firms’ Website Interfaces and Performance. Electronics 2024 , 13 , 3256. https://doi.org/10.3390/electronics13163256

Giannakopoulos NT, Sakas DP, Migkos SP. Neuromarketing and Big Data Analysis of Banking Firms’ Website Interfaces and Performance. Electronics . 2024; 13(16):3256. https://doi.org/10.3390/electronics13163256

Giannakopoulos, Nikolaos T., Damianos P. Sakas, and Stavros P. Migkos. 2024. "Neuromarketing and Big Data Analysis of Banking Firms’ Website Interfaces and Performance" Electronics 13, no. 16: 3256. https://doi.org/10.3390/electronics13163256

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals