STAT 5374 -- Theory of Linear Statistical Models -- Fall 2017
Trindade, 228 Mathematics & Statistics Building.
Course Meets: 11:00 - 12:20 TR in MATH 115.
Office Hours (in 201): Tue/Thurs 9:00-11:00, 13:00-15:00, or by appointment.
Linear Models in Statistics, by Rencher & Schaalje, 2nd edition, 2008, Wiley. ISBN-13: 978-0471754985.
- Linear Model Methodology. Khuri, A. (2010). CRC Press.
- Plane Answers to Complex Questions: The Theory of Linear Models. Christensen, R. (1996). Second Edition. Springer.
- Linear Statistical Models. Stapleton, J.H. (2009). Second Edition. Wiley.
- Linear Models for Unbalanced Data. Searle, S.R. (1987). Wiley.
- Theory and Application of the Linear Model. Graybill, F.A. (1976). Duxbury Press.
- Linear Models. Searle, S.R. (1971). Wiley.
- Theory of matrices
- A Matrix Handbook for Statisticians, Seber, G.A.F. (2007). Wiley.
- Matrix Algebra From a Statistician's Perspective, Harville, D.A. (1997). Springer.
- Handbook of Matrices, Lutkepohl, H. (1996). Wiley.
- Matrices with Applications in Statistics. Graybill, F.A., 2nd ed., (1983). Wadsworth.
Course Objectives and Syllabus
Classical linear models are at the core of statistics, and are
the most commonly used set of statistical techniques in practice.
The two main subclasses of the classical linear model are
(1) linear regression models, and (2) analysis of variance (ANOVA) models.
A third subclass, (3) analysis of covariance models (ANCOVA) models,
combines elements of regression and ANOVA.
Because these models are such important practical tools for data analysis,
instruction in the practical aspects of their application is a crucial part of
a statistician's training. However, these methods are not a collection
of unrelated, specialized techniques. This course will cover the general theory of estimation and
inference in the linear model, which includes regression, ANOVA, and ANCOVA as special cases. This corresponds to chapters 1-9 and 12-14 of the book. Prerequisite
: STAT 5329 (Math-Stat). The list of topics is as follows:
This theory is worth studying because it unifies and yields insight into the
methods used in many important subcases of the linear model; and
because its ideas point the way and, in some cases, carry over directly to
more general (not-necessarily-linear) modeling of data.
In summary, this is a theory course, and as such it is not a complete course
in linear models. Very important practical aspects of these models will be
covered in other courses.
- Concepts from linear algebra: matrices; vectors; matrix algebra; inner products; orthogonal projections; eigenvalues and eigenvectors; systems of equations; generalized inverses.
- Random vectors & matrices: quadratic forms; the multivariate normal and associated chi-square, t, and F distributions.
- The linear model (full rank case): regression; Gauss-Markov Theorem & best linear unbiased estimators (BLUEs); OLS & GLS; statistical inference.
- The linear model (non-full rank case): ANOVA models (one-way & two-way).
- The linear mixed model (time permitting).
Expected Student Learning Outcomes
By the end of the course, students will be expected to become adept at solving problems that form part of the syllabus for the Applied Statistics Preliminary Examination. These include: linear algebra computations relating to vectors, matrices, projection operators, eigenvalues and eigenvectors, systems of equations, and generalized inverses. Calculations pertaining to random vectors & matrices, quadratic forms, and the multivariate normal and associated chi-square, t, and F distributions. The primary goal of the course is to cover the theory of statistical inference for the linear model in both the full rank and less than full rank cases (for the design matrix). Thus knowledge of the properties and theorems relating to inference for model coefficients under ordinary least squares, generalized least squares, and maximum likelihood estimation, will be at the heart of the expected learning outcomes.
Methods of Assessing the Expected Learning Outcomes
The expected learning outcomes for the course will be assessed through a mix of homework assignments (10%), two midterm tests (25% each), and a comprehensive final exam (40%). The traditional grading scale will be used:
The test schedule is as follows:
- A: 90-100%.
- B: 80-89%.
- C: 70-79%.
- D: 60-69%.
- F: 0-59%.
- Test 1: Thursday October 5.
- Test 2: Thursday November 9.
- Final Exam: Friday, December 8, 1:30 - 4:00 pm.
There will be weekly Assignment Sets. All work handed in must be stapled together. No late submissions will be accepted.
- Set 0 (due Thurs Aug 31): 2.2, 2.4, 2.15, 2.17, 2.18, 2.19.
- Set 1 (due Thurs Sep 7): Hwk 1.
- Set 2 (due Thurs Sep 14): Hwk 2.
- Set 3 (due Thurs Sep 21): Hwk 3.
- Set 4 (due Thurs Sep 28): Hwk 4.
- Test 1 (Thurs Oct 5).
- Set 5 (due Thurs Oct 12): Hwk 5.
- Set 6 (due Thurs Oct 19): Hwk 6.
- Set 7 (due Thurs Oct 26): Hwk 7.
- Set 8 (due Thurs Nov 2): Hwk 8.
- Test 2 (Thurs Nov 9).
- Set 9 (due Thurs Nov 16): 7.53 (Gas Vapor Data), 8.5, 8.6, 8.8, 8.12, 8.18, 8.19, 8.25.
- Set 10 (due Tue Nov 28): handout.
- Set 11 (due Tue Dec 5): handout.
I will use R as the primary software tool. SAS is also recommended. Some assignments will require extensive use of a software package of your choice. For details on R see my statistical computing page, and especially the section on "Linear Models & GLMs".
- Collaboration. My specific policies are as follows.
- Homeworks: Discussion with peers regarding material/concepts covered in the
course is permitted, and is encouraged since it usually leads to greater comprehension. However, each person must write up his/her own
solution to a particular problem, and not simply copy it from someone else.
- Tests: Any form of collaboration in tests, including trying to see
what the person next to you is writing, is strictly forbidden and will not be tolerated.
- Class Attendance. Your attendance alone will not impact your grade,
but missing exams and assignments will. Whether an absence is excused or
unexcused is determined solely by me, with the exception of absences due to
religious observance and officially approved trips (see below).
- Make-up Exams: These may be granted in exceptional circumstances
if you provide me with a valid excuse (such as a note from a physician, an
- Absence for observance of a religious holy day (TTU Operating Policy 34.19): 1. "Religious holy day" means a holy day observed by a religion whose places of worship are exempt from property taxation under Texas Tax Code 11.20. A student who intends to observe a religious holy day should make that intention known in writing to the instructor prior to the absence. A student who is absent from classes for the observance of a religious holy day shall be allowed to take an examination or complete an assignment scheduled for that day within a reasonable time after the absence. A student who is excused may not be penalized for the absence; however, the instructor may respond appropriately if the student fails to complete the assignment satisfactorily.
- Absence due to officially approved trips: The Texas Tech University Catalog states that the department chairpersons, directors, or others responsible for a student representing the university on officially approved trips should notify the student's instructors of the departure and return schedules in advance of the trip. The instructor so notified must not penalize the student, although the student is responsible for material missed. Students absent because of university business must be given the same privileges as other students.
- Illness and Death Notification. The Center for Campus Life is responsible for notifying the campus community of student illnesses, immediate family deaths and/or student death. Generally, in cases of student illness or immediate family deaths, the notification to the appropriate campus community members occur when a student is absent from class for four (4) consecutive days with appropriate verification. It is always the student's responsibility for missed class assignments and/or course work during their absence. The student is encouraged to contact the faculty member immediately regarding the absences and to provide verification afterwards. The notification from the Center for Campus Life does not excuse a student from class, assignments, and/or any other course requirements. The notification is provided as a courtesy.
- ADA accommodations (TTU Operating Policy 34.22). Any student who, because of a disability, may require special arrangements in order to meet the course requirements should contact the instructor as soon as possible to make any necessary arrangements. Students should present appropriate verification from Student Disability Services during the instructor's office hours. Please note: instructors are not allowed to provide classroom accommodations to a student until appropriate verification from Student Disability Services has been provided. For additional information, please contact Student Disability Services in West Hall or call 806-742-2405.
- Civility in the Classroom. It is expected that everyone will behave
in a manner that is conducive to learning. One common disruption is phones. Please turn these off in class.
- Academic Honesty (TTU Operating Policy 34.12). It is the aim of the faculty of Texas Tech University to foster a spirit of complete honesty and high standard of integrity. The attempt of students to present as their own any work not honestly performed is regarded by the faculty and administration as a most serious offense and renders the offenders liable to serious consequences, possibly suspension. "Scholastic dishonesty" includes, but it not limited to, cheating, plagiarism, collusion, falsifying academic records, misrepresenting facts, and any act designed to give unfair academic advantage to the student (such as, but not limited to, submission of essentially the same written assignment for two courses without the prior permission of the instructor) or the attempt to commit such an act.
top of page