United States District Court, District of Massachusetts
May 7, 2015
MASSACHUSETTS MUTUAL LIFE INSURANCE COMPANY, Plaintiff,
DB STRUCTURED PRODUCTS, INC., et al. Defendants.
MEMORANDUM AND ORDER REGARDING DEFENDANTS’ MOTIONS TO EXCLUDE THE EXPERT TESTIMONY OF JOHN A. KILPATRICK AND STEVEN I. BUTLER (Dkt. Nos. 355 and 359)
MARK G. MASTROIANNI United States District Judge
Massachusetts Mutual Life Insurance Company (“Plaintiff”) brought eleven related actions against various defendants, asserting violations of Mass. Gen. Laws ch. 110A, § 410, the Massachusetts Uniform Securities Act (“MUSA”), for misstatements and omissions contained in the offering documents of certain residential mortgage-backed securities (“RMBS”). The instant action (11-cv-30039-MGM), brought against Deutsche Bank Securities Inc. (“DBSI”), Anilesh Ahuja, Michael Commaroto, Richard D’Albert, and Richard Ferguson (together, “Defendants”), was designated a “bellwether” case by Judge Saris on December 4, 2013. (See Dkt. No. 225.) Accordingly, it is scheduled to proceed through summary judgment and trial while the other cases are stayed.
Presently before the court are Defendants’ motions to exclude the expert testimony of John A. Kilpatrick and Steven I. Butler, under Federal Rule of Evidence 702 and Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579 (1993). For the following reasons, the court will grant in part and deny in part Defendants’ motion to exclude Dr. Kilpatrick, and deny the motion to exclude Mr. Butler.
Plaintiff brought these eleven actions in 2011, alleging that material misrepresentations were made in the sale of RMBS in violation of MUSA. RMBS are securities entitling the holder to income payments from pools of residential mortgage loans held by a trust. After the various mortgage loans are aggregated into loan pools and securitized, certificates representing the securitizations are sold to investors. Accordingly, the characteristics of the loans underlying the certificates, such as the loan-to-value (“LTV”) and debt-to-income (“DTI”) ratios, as well as the loan origination and underwriting standards,  affect the certificates’ value. In this case, Plaintiff alleges the offering materials for the certificates contained misrepresentations regarding (1) the underwriting guidelines and standards used to originate or acquire the mortgage loans underlying the certificates; (2) the appraisal standards used to value the mortgaged properties; and (3) the LTV ratios for the underlying loans.
On July 21, 2014, Plaintiff served the affirmative expert reports of Dr. Kilpatrick and Mr. Bulter, among others. Dr. Kilpatrick prepared two expert reports: in one, he assessed the accuracy of the original appraised values of a sample of properties underlying the certificates (“Sample Properties”) using an automated valuation model (“AVM Report”); in the other, he evaluated, for the subset of properties in the AVM Report which he found were significantly overvalued, whether the appraisals adhered to applicable appraisal standards using a credibility assessment model (“CAM Report”). Mr. Butler, for his part, prepared a report in which he analyzed, or “reunderwrote, ” the loans of the Sample Properties to determine whether they were originated or acquired in accordance with applicable underwriting guidelines in compliance with representations in the offering documents (“Butler Report”).
A. Dr. Kilpatrick
Dr. Kilpatrick has over 30 years of experience in finance and financial analysis, real estate, business development, statistical analysis, consulting, and teaching. (Dkt. No. 503, Decl. of John A. Kilpatrick (“Kilpatrick Decl.”) ¶ 6; Ex. 1 (“AVM Report”) at 3.) He earned a Bachelor of Science in business administration (accounting), a Master of Business Administration, and a Ph.D in real estate finance from the University of South Carolina. (Kilpatrick Decl. ¶ 7; AVM Report at 5; Ex. 3 at 2.) Dr. Kilpatrick currently teaches real estate finance as a visiting scholar at Baruch College, City University of New York. (Kilpatrick Decl. ¶ 7; AVM Report at 5.) In addition, he is an MAI designated member of the Appraisal Institute and is a state-certified (general) real estate appraiser in all 50 states as well as the District of Columbia. (Kilpatrick Decl. ¶ 8; AVM Report at 5.) In 2004, the Appraisal Qualifications Board (“AQB”) designated Dr. Kilpatrick as one of approximately 500 nationally certified appraisal standards instructors. (Kilpatrick Decl. ¶ 9; AVM Report at 6.) Dr. Kilpatrick also was nominated to the AQB and was named a Fellow of the Faculty of Valuation of the British Royal Institution of Chartered Surveyors. (Id.) He has authored numerous books and articles on real estate and serves as an editorial board member of two peer-reviewed publications: the Journal of Sustainable Real Estate and The Appraisal Journal. (Kilpatrick Decl. ¶ 10; AVM Report at 6.)
With regard to automated valuation models in particular, Dr. Kilpatrick has authored or co-authored several peer-reviewed publications on the subject. (Kilpatrick Decl. ¶ 12.) In addition to teaching, Dr. Kilpatrick is the Managing Director of Greenfield Advisors LLC, a real estate appraisal and economic consulting firm headquartered in Seattle, Washington. (AVM Report at 6.) Dr. Kilpatrick and the Greenfield Advisors staff have written and lectured extensively on the application of mass appraisal and statistical methods in class action litigation. (Id. at 6-7.) Moreover, Dr. Kilpatrick has testified as an expert in 41 actions in the past four years for both plaintiffs and defendants, and he has been qualified as a mass appraisal and real estate valuation expert in a number of cases. (Kilpatrick Decl. ¶ 13; AVM Report at 7; Ex. 5.)
Dr. Kilpatrick’s AVM Report describes his use of the Greenfield Automated Valuation Model (“GAVM”), an AVM he previously developed but adapted for this action, to assess the accuracy of the original appraised values of a sample of properties underlying the certificates. AVMs are computer programs that use statistical models to reach objective estimates of the market value of real property. (AVM Report at 27.) GAVM in particular uses a statistical hedonic regression model to perform retrospective real estate appraisals. (Kilpatrick Decl. ¶ 15.) Similar to a traditional appraisal, GAVM uses the sales prices of comparable properties to generate a value for the subject property. (Id. ¶ 18.) Unlike a traditional appraisal, however, GAVM uses 100 to 2, 000 sales transactions as comparables, rather than only three or six. (Id. ¶ 17; AVM Report at 30.) These comparable sales transactions are from the same county as the subject property, and the model limits the sales observation data to the geographically nearest sales transactions in the year preceding the effective date of the Sample Property’s appraisal. (Kilpatrick Decl. ¶ 19; AVM Report at 32.) As Dr. Kilpatrick explains in his AVM Report, “[b]ecause it relies on a larger number of comparisons and uses uniform computing logic, an AVM’s valuation results are inherently more accurate and objective” than a traditional appraisal, which is prone to subjectivity in selecting comparable properties and adjusting sales data. (AVM Report at 29-30.)
GAVM consists of two valuation sub-models: an ordinary least squares regression model (“OLS valuation model”), and an OLS model with a trend surface component (“OLSXY valuation model”). (Kilpatrick Decl. ¶ 19; AVM Report at 32.) The use of the trend surface component allows the model to incorporate spatial effects, i.e., value impacts due to location, into the valuation. (Id.) The required variables for the OLS model are tax assessed value, days between an origin date and sale, and days between an origin date and sale squared. (Kilpatrick Decl. ¶ 20; AVM Report at 41.) The OLS model can also accept home size, year built, lot size, and number of bathrooms, if sufficient data about these variables are available. (Id.) The required variables for the OLSXY model are tax assessed value, days between an origin date and sale, days between an origin date and sale squared, latitude, longitude, latitude squared, longitude squared, and latitude multiplied by longitude. (Kilpatrick Decl. ¶ 21; AVM Report at 41-42.) The OLSXY model can also accept home size, year built, lot size, and number of bathrooms, again, if these variables are available. (Id.) The data used to run the models was obtained from CoreLogic, which maintains the largest, most comprehensive property databases in the United States, as well as the loan tapes and loan files for the Sample Properties. (Kilpatrick Decl. ¶ 22; AVM Report at 20-22.) This data included previous sales prices, previous sales data, property characteristics such as footage, baths, age, and lot size, and location on a county-by-county basis, from January 1, 1998 through December 31, 2008. (Kilpatrick Decl. ¶ 22; AVM Report 22-23.)
Before running the Sample Properties through GAVM, Dr. Kilpatrick both calibrated and filtered the model using only the comparable properties. First, Dr. Kilpatrick “calibrated” GAVM, or tested its accuracy, using approximately 1.53 million sales from throughout the country. (Kilpatrick Decl. ¶¶ 26-27; AVM Report at 48.) For each of the 2, 783 counties in the dataset, he randomly split sales into a 10% set and a 90% set. (Id.) Then, he ran the 10% set through GAVM using the 90% set as the comparables. (Kilpatrick Decl. ¶ 28; AVM Report at 48.) Dr. Kilpatrick removed the following from his calibration analysis: transactions where the home was constructed after the sales date, because this indicated the sale was likely a land sale; transactions outside the middle 70th percentile of sales price to assessed value ratio, to eliminate suspect data and/or non-arm’s-length transactions; and properties with “errors” over 100%, because these likely indicated data errors. (Kilpatrick Decl. ¶ 28; AVM Report at 48-49.) Using his 10% set, Dr. Kilpatrick calculated GAVM’s coverage, or “hit rate, ” which measures how many of the properties the model was able to value, as well as the forecast standard deviation (“FSD”), which measures the model’s ability to predict known sales prices. (Kilpatrick Decl. ¶¶ 30-31; AVM Report at 50-52.) GAVM’s hit rate, without applying the 100% error filter, was 98.0%, which is much higher than the 80% hit rate Freddie Mac has noted is considered high in the industry. (Kilpatrick Decl. ¶ 30; AVM Report at 51.) Regarding FSD, of the approximately 1.53 million sales transactions valued by GAVM, 72.7% of the valuations were within 13% of the actual sales prices. (Kilpatrick Decl. ¶ 34; AVM Report at 52.) The FSD for GAVM was 15.1%, and the mean absolute error was 10.45%. (Id.)
Second, Dr. Kilpatrick used a “Cross-Validation Filter” to ensure that the comparable properties used in GAVM were properly valued. (Kilpatrick Decl. ¶ 42.) This filter eliminated comparable properties from GAVM if the sales price was at least 25% higher or lower than estimated using GAVM based on the remaining comparable properties in the same county. (Id.; AVM Report at 59-60.) Dr. Kilpatrick used this filter because he concluded that such properties either were not sold in arm’s-length transactions or had certain characteristics that significantly impacted the sales price which either were not knowable using the available data or which the GAVM model did not use. (Kilpatrick Decl. ¶ 42.) Accordingly, the Cross-Validation Filter was an attempt to remove outlier comparable properties which would unduly skew the analysis. (Id. ¶¶ 42-43.)
Dr. Kilpatrick then ran 924 of the 998 Sample Properties through both the OLS and OLSXY sub-models of GAVM. (AVM Report at 35-37, 60-62.) If only one sub-model produced a value, then that value constituted the final GAVM value prediction for the Sample Property, but if both sub-models produced a value, then the two values were averaged together to produce a final value prediction. (Id. at 35-36.) Using this process, Dr. Kilpatrick found the original appraised values of 211 of the 924 Sample Properties were more than 15.1% (the FSD discussed above) higher than their true, credible appraised values. (Kilpatrick Decl. ¶ 47; AVM Report at 62 n.158.) In addition, Dr. Kilpatrick found the original appraisals for the Sample Properties were systemically overvalued by a statistically significant average of 6.40% and that the 10% of supporting loan groups which were overvalued the most were overvalued by an average of 9%. (Kilpatrick Decl. ¶¶ 3, 47; AVM Report at 4, 63.) From this, Dr. Kilpatrick concluded that the LTV ratios for the Sample Properties were significantly higher than represented in the offering documents for the securitizations. (Id.) Dr. Kilpatrick further concluded that “no reasonable, competent appraisal professional adhering to appraisal standards applicable at the time could, in my opinion, validate the Deutsche Bank Appraisals as reliable, unbiased, or accurate.” (Kilpatrick Decl. ¶ 58; AVM Report at 66.)
In addition to his GAVM Report, Dr. Kilpatrick also prepared a CAM Report, in which he assessed the appraisals for 206 of the 211 Sample Properties identified in the GAVM Report as significantly overvalued. (Kilpatrick Decl. ¶¶ 59-60; Ex. 2 (“CAM Report”) at 2.) The purpose of the CAM Report was to determine whether these appraisals were “credible, ” i.e., whether they adhered to appraisal standards and practices pursuant to the Uniform Standards of Professional Appraisal Practice (“USPAP”). (Id.) To do this, Dr. Kilpatrick developed a Credibility Assessment Model (“CAM”) which evaluates the frequency and magnitude of an appraisal report’s deviation from settled appraisal standards and practice. (Kilpatrick Decl. ¶ 61; CAM Report at 2-3.) The CAM scores appraisals using a list of 31 questions based on USPAP and other professional guidance applicable to residential appraisals at the time. (Kilpatrick Decl. ¶ 67; CAM Report at 40.) The questions are phrased to elicit a “yes” or “no” response based on a review of information derived from loan tapes, loan files, and third-party sources. (Id.) A “no” response, which indicates that an appraisal process was not properly followed, is also given a specific weight or score based on the degree to which it affects the credibility of the appraisal. (Kilpatrick Decl. ¶ 69; CAM Report at 40 41.) Thus, a “no” response to one question may result in a score of 2.54, while a “no” response to another question may result in a score of 9.2. (Kilpatrick Decl. ¶ 72; CAM Report at 72, 89; Ex. 28.)
Dr. Kilpatrick also developed a scoring threshold, based on USPAP Standards Rules 1-1(b) and 1-1(c),  beyond which he concluded the adherence to applicable appraisal standards was so poor as to render the appraisal not “credible.” (Kilpatrick Decl. ¶ 74; CAM Report at 41, 98.) He concluded that a total score of 14.13 could render an appraisal not credible under USPAP due to either one major and one minor “no” response or a series of minor “no” responses. (Kilpatrick Decl. ¶ 76; CAM Report at 99.) In order to evaluate the appraisals in the most conservative light, Dr. Kilpatrick then raised this threshold score to 20, which means that an appraisal must elicit at least three “no” responses, two of which would have to be major questions, to be deemed not credible under the CAM. (Kilpatrick Decl. ¶ 80; CAM Report at 100-101.)
Of the 206 Sample Properties found to be overvalued by more than 15.1% using GAVM, Dr. Kilpatrick found 168 appraisals (81.55%) were not credible using CAM. (Kilpatrick Decl. ¶ 80; CAM Report at 101.) Dr. Kilpatrick explained that these appraisals contain sufficient errors such that, in his opinion, “no reasonable appraiser adhering to the appraisal standards and practices at the time could have believed the appraisals were credible.” (Kilpatrick Decl. ¶ 80; CAM Report at 101.) On average, the appraisals contained 6.38 errors, or “no” responses. (Kilpatrick Decl. ¶ 81; CAM Report at 102.)
In addition to using the CAM, Dr. Kilpatrick also had Recovco Mortgage Management, LLC (“Recovco”), a separate entity, independently review 205 of the appraisals for the Sample Properties found by GAVM to be significantly overvalued. (Kilpatrick Decl. ¶ 82; CAM Report at 103.) Recovco performed its standard collateral review to evaluate whether the appraisals adhered to USPAP, Fannie Mae, and industry standard appraisal guidelines. (Kilpatrick Decl. ¶ 83; CAM Report at 103.) Based on its evaluation, Recovco rendered a conclusion as to whether the appraisal was “weak” (not credible), “moderate” (of questionable credibility), or “strong” (credible). (Kilpatrick Decl. ¶ 83; CAM Report at 104.) It found 95.1% of the 205 appraisals were “weak” and demonstrated significant discrepancies indicative of a non-credible appraisal. (Kilpatrick Decl. ¶ 86; CAM Report at 104.) Recovco found the remaining appraisals were “moderate.” (Kilpatrick Decl. ¶ 87.) According to Dr. Kilpatrick, Recovco’s review “validated” his CAM findings. (Id. ¶ 88.)
B. Mr. Butler
Mr. Butler has over 43 years of experience in residential mortgage lending, including fifteen years of experience underwriting and originating mortgage loans and supervising the origination and approval of mortgage loans at multiple banks. (Dkt. No. 499, Decl. of Steven I. Butler (“Butler Decl.”) ¶ 2.) He has also worked as a consultant in the banking industry since 1986. (Id.) This work has included reviewing, or “reunderwriting, ” residential mortgage loans to determine whether they were originated and underwritten in accordance with applicable underwriting guidelines and industry standards. (Id.; Ex. 1 (“Butler Report”) at 8-11.) In addition, Mr. Butler has served as an expert in a number of cases, including as a mortgage loan reunderwriting expert in ten RMBS cases since 2009. (Butler Decl. ¶ 5; Butler Report at 12.)
In his Report, Mr. Butler reunderwrote loan files underlying certificates at issue in this action to determine whether they complied with representations in the offering documents, including representations regarding compliance with applicable underwriting guidelines and the borrowers’ ability to repay. (Butler Report at 1-2.) Using the representative Sample Properties selected by Dr. Cowan, Mr. Butler reunderwrote loans for which files were available that contained over 100 pages.
(Id. at 115 n.343.) If a loan file was not produced or contained fewer than 100 pages, Mr. Butler used a loan from the supplemental random sample of 100 loans for each supporting loan group, in accordance with Dr. Cowan’s sampling methodology. (Id.) See Massachusetts Mut. Life Ins. Co., 989 F.Supp.2d at 170. Mr. Butler ended up using 167 such supplemental loans and reunderwrote a total of 998 loans. (Id.)
Before reunderwriting the loans, Mr. Butler instructed his team of experienced underwriters to match the loans with the applicable underwriting guidelines in effect at or near the time of loan origination whenever they were available. (Butler Decl. ¶ 13; Butler Report at 116.) If the applicable originator underwriting guidelines were not available, however, he instructed the reunderwriting team to use the applicable aggregator underwriting guidelines-i.e., underwriting guidelines of an entity other than the originator, such as the securitization sponsor or an affiliate of the sponsor-in effect at or near the time of origination. (Id.) Mr. Butler explained that, based on his experience, “aggregator guidelines are appropriate to use when the originator guidelines are not available because it is the aggregator that acquires the loans that are to be securitized and, in many instances, either approves the guidelines used by the originator or instructs the originator to use the aggregator’s guidelines” and, “[t]ypically, the guidelines of the originator and the aggregator have similar parameters and metrics.” (Butler Decl. ¶ 13.)
If neither the applicable originator nor aggregator underwriting guidelines were available, or if the applicable guideline was silent on an issue,  Mr. Butler directed his team to use “Minimum Industry Standards” to reunderwrite the loan file. (Butler Decl. ¶ 15; Butler Report at 111.) The Minimum Industry Standards, Mr. Butler explained, were basic industry standards which constituted lenient standards found in underwriting guidelines in the mortgage industry from 2005 to 2007. (Butler Decl. ¶ 16; Butler Report at 112.) “In other words, they were the minimum requirements necessary for even the most lenient of originators to assess a borrower’s ability to repay the mortgage and adequacy of the collateral.” (Butler Report at 112.) The Minimum Industry Standards include verifying employment, investigating potential borrower misrepresentations, and taking steps to ensure the borrower has the ability to repay the loan, among others. (Butler Decl., Ex. 8.)
Mr. Butler gleaned the Minimum Industry Standards from (1) his experience in the industry and knowledge of standards used from 2005 to 2007; (2) discussions with underwriters on his staff and members of the reunderwriting team who worked in the mortgage loan industry at that time; (3) a review of underwriting guidelines, including manuals, references, matrices, and guides, used by originators during that time; and (4) discussions with other experts he knows and with whom he worked. (Butler Decl. ¶ 17; Butler Report at 112.) To confirm the accuracy of the Minimum Industry Standards, Mr. Butler compared them to the guideline requirements of four significant originators of loans between 2005 and 2007 that were known in the industry as having relatively lenient underwriting standards: Countrywide Home Loans, Inc., Long Beach Mortgage Company, New Century Financial Corporation, and WMC Mortgage Corp. (Butler Decl. ¶ 18; Butler Report at 112.) He determined that the Minimum Industry Standards were generally consistent with these four originators’ guidelines. (Id.)
Mr. Butler and his team found 3, 539 individual underwriting guideline breaches in the 998 reunderwritten loans; most loans had multiple breaches. (Butler Decl. ¶ 7.) He determined that 744 of the 998 loans were originated with one or more breaches that meaningfully and substantially increased the credit risk associated with the loan. (Id. ¶ 7; Butler Report at 4.) The breaches included LTV or DTI ratios that exceeded guidelines; absence of necessary documentation; failure to investigate red flags, such as multiple loan applications with different stated incomes, recent credit inquiries, and high stated income with low assets and bank deposits; and failure to investigate the reasonableness of stated income. (Butler Decl. ¶ 7; Butler Report at 131-148.) Of those 744 materially defective loans, 475 (over 65%) were reunderwritten using the originators’ guidelines; 205 (over 27%) were reunderwritten using the aggregator’s guidelines; and 64 (8.6%) were reunderwritten using either the Minimum Industry Standards only or Minimum Industry Standards in combination with other sources such as representations in the offering documents. (Butler Decl. ¶¶ 14, 20.)
C. Procedural History
Defendants filed the instant motions to exclude the expert testimony of Dr. Kilpatrick and Mr. Butler (“Daubert Motions”) on November 21, 2014. Plaintiff filed its oppositions on December 22, 2014 and included new declarations from Dr. Kilpatrick and Mr. Butler in support. On January 15, 2015, Defendants filed, along with their reply briefs, a motion to strike the declarations pursuant to Rules 26(a)(2) and 37(c)(1) on the grounds that they included new and previously undisclosed opinions. On March 27, 2015, the court held a hearing on the motion to strike as well as a non-evidentiary hearing on the Daubert Motions, as requested by the parties. At the hearing, the court inquired as to the possibility of re-opening discovery regarding any “new” material included in the declarations and the effect that might have on the Daubert Motions. Defendants’ counsel represented that regardless of the court’s resolution of the motion to strike, including any re-opening of discovery, Defendants wished to go forward with the Daubert Motions. On March 31, 2015, the court denied Defendants’ motion to strike, concluding most of the content in the declarations constituted proper expert supplementation in response to new criticisms raised after the experts provided their original reports. (See Dkt. No. 439.) The court did, however, re-open discovery on a limited number of issues which it found to constitute improper expert supplementation, but explained that it would not await the close of the new expert discovery before ruling on the Daubert Motions in light of Defendants’ counsel’s representation at the March 27, 2015 hearing. (See id.)
III. Standard of Review
The admission of expert evidence is governed by Federal Rule of Evidence 702, which codified the Supreme Court's holding in Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579 (1993), and its progeny. See United States v. Diaz, 300 F.3d 66, 73 (1st Cir. 2002). Rule 702 provides:
If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.
Fed. R. Evid. 702. The trial court must determine whether the expert's testimony “both rests on a reliable foundation and is relevant to the task at hand” and whether the expert is qualified. Daubert, 509 U.S. at 597; Diaz, 300 F.3d at 73. An expert's methodology is the “central focus of a Daubert inquiry.” Ruiz-Troche v. Pepsi Cola of P.R. Bottling Co., 161 F.3d 77, 81 (1st Cir. 1998). “In short, a court must answer three questions: ‘Is this junk science?’ ‘Is this a junk scientist?’ Is this a junk opinion?’” First Choice Armor & Equipment, Inc. v. Toyobo America, Inc., 839 F.Supp.2d 407, 416 (D. Mass. 2012) (quoting McGovern ex rel. McGovern v. Brigham & Women’s Hosp., 584 F.Supp.2d 418, 424 (D. Mass. 2008)).
The Daubert Court identified four factors which might assist a trial court in determining the admissibility of an expert’s testimony: (1) whether the theory or technique can be and has been tested; (2) whether the technique has been subject to peer review and publication; (3) the technique's known or potential rate of error; and (4) the level of the theory's or technique's acceptance within the relevant discipline. Milward v. Acuity Specialty Products Group, Inc., 639 F.3d 11, 14 (1st Cir. 2011). “These factors, ” however, “‘do not constitute a definitive checklist or test.’” Id. (quoting Kumho Tire Co. v. Carmichael, 526 U.S. 137, 150 (1999)). “Given that ‘there are many different kinds of experts, and many different kinds of expertise, ’ these factors ‘may or may not be pertinent in assessing reliability, depending on the nature of the issue, the expert’s particular expertise, and the subject of his testimony.’” Id. (quoting Kumho Tire Co., 526 U.S. at 150.)).
While expert testimony may be excluded if there is “too great an analytical gap between the data and the opinion proffered, ” id. at 15 (quoting Gen. Elec. Co. v. Joiner, 522 U.S. 136, 146 (1997)), “[t]his does not mean that trial courts are empowered ‘to determine which of several competing scientific theories has the best provenance, ’” id. (quoting Ruiz-Troche, 161 F.3d at 85.)). “Daubert does not require that a party who proffers expert testimony carry the burden of proving to the judge that the expert’s assessment of the situation is correct.” Id. (quoting Ruiz-Troche, 161 F.3d at 85)). Rather, “[t]he proponent of the evidence must show only that ‘the expert’s conclusion has been arrived at in a scientifically sound and methodologically reliable fashion.’” Id. (quoting Ruiz-Troche, 161 F.3d at 85)). As long as an expert’s scientific testimony rests upon “‘good grounds, ’ based on what is known, ” Daubert, 509 U.S. at 590, “it should be tested by the adversarial process, rather than excluded for fear that jurors will not be able to handle the scientific complexities, ” Milward, 639 F.3d at 15. Thus, “[v]igorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence.” Daubert, 509 U.S. at 596.
A. Motion to Exclude Dr. Kilpatrick’s Testimony
Defendants seek to exclude Dr. Kilpatrick’s testimony on a number of grounds. They challenge his qualifications as an expert and the lack of independent validation, peer review or publication, and industry acceptance of GAVM and CAM. In addition, regarding GAVM, Defendants assert it overstates LTV; is not an appraisal and, thus, is being misused by Dr. Kilpatrick; improperly ignores sales data; produces inconsistent results between the OLS and OLSXY sub-models; improperly uses tax assessed value as a variable, but omits other important variables; and employs dubious filtering techniques. Regarding CAM, Defendants assert Dr. Kilpatrick improperly opines as to the original appraisers’ state of mind, bases his analysis on unreliable information, provides no support for the 31 questions or weighting used, and improperly relies on Recovco’s analysis. In response, Plaintiff asserts Defendants’ attacks on GAVM and CAM are meritless and, at most, go to the weight, rather than the admissibility, of Dr. Kilpatrick’s opinions.
As an initial matter, the court finds Dr. Kilpatrick is sufficiently qualified as an expert. He has extensive experience regarding real estate appraisals and automated valuation models. Defendants have not cited a single case in which a court has excluded Dr. Kilpatrick’s testimony based on a lack of expertise or qualification, despite numerous Daubert challenges. Moreover, while Defendants point to Dr. Kilpatrick’s false statements on his Certified Real Estate Appraiser applications for two states and his purported difficulty testifying at his deposition without referring to his materials, these are, at most, issues of credibility for the jury and do not warrant excluding Dr. Kilpatrick’s testimony. See, e.g., Seahorse Marine Supplies, Inc. v. Puerto Rico Sun Oil Co., 295 F.3d 68, 81 (1st Cir. 2002) (“The ultimate credibility determination and the testimony’s accorded weight are in the jury’s province.”).
As for Dr. Kilpatrick’s testimony regarding GAVM, the court finds his opinion is relevant and rests upon sufficiently reliable grounds for admissibility purposes. First, contrary to Defendants’ assertions, peer review and independent validation are not necessarily required. See Daubert, 509 U.S. at 593 (explaining that the four identified factors are not “a definitive checklist or test” and noting that peer review and publication “does not necessarily correlate with reliability”); Granfield v. CSX Transp., Inc., 597 F.3d 474, 486 (1st Cir. 2010) (“The mere fact of publication, or lack thereof, in a peer-reviewed journal is not a determinative factor in assessing the scientific validity of a technique or methodology on which an opinion is premised.”). In any event, it is clear that AVMs have been subject to peer review and are accepted within the real estate appraisal and underwriting industry, although this exact model is particular to Dr. Kilpatrick. (Kilpatrick Decl. ¶ 12, Ex. 4.) See Assured Guar. Mun. Corp. v. Flagstar Bank, FSB, 920 F.Supp.2d 475, 505 (S.D.N.Y. 2013) (noting that the methodology applied by the plaintiff’s expert, which included the use of an AVM, “is the same kind of methodology underwriters apply in the field”); In re Katrina Canal Breaches Consol. Litig., 2007 WL 3245438, at *13 (E.D. La. Nov. 1, 2007) (“Clearly, mass appraisal is an accepted methodology. . . . This approach is used in appraiser’s offices all over the country to determine value.” (internal citation omitted)); Turner v. Murphy Oil USA, Inc., 2006 WL 91364, at *5 (E.D. La. Jan. 12, 2006) (“Dr. Kilpatrick is qualified and is using generally-accepted methodology.”). In fact, Deutsche Bank itself used AVMs as part of its valuation due diligence for loans at issue in this action. (Dkt. No. 507, Decl. of Jennifer J. Barrett (“Barrett Decl.”), Ex. 20 at 84, Ex. 21 at 99.) Moreover, as Plaintiff’s counsel explained at the hearing, unlike commercially available AVMs, which are proprietary and thus cannot be tested or reverse-engineered, Dr. Kilpatrick provided the computer code for GAVM and the CoreLogic database to Defendants, whose experts were able to replicate its results. (Barret Decl., Ex. 5 at 59-60, Ex. 10.) Accordingly, Dr. Kilpatrick’s methodology can be adequately tested on cross examination. See Assured Gaur. Mun. Corp., 920 F.Supp.2d at 505.
Defendants’ other arguments for excluding Dr. Kilpatrick’s opinion regarding GAVM all go to weight rather than admissibility. In particular, while Defendants argue Dr. Kilpatrick improperly recalculated the LTV ratios for the Sample Properties by using the lower of the GAVM value, sales prices, or original appraised value, he did so because the Prospectus Supplements stated LTV ratios were calculated in this manner. (Kilpatrick Decl. ¶ 52; AVM Report at 67 n.164; Dkt. No. 374, Ex. 1 at 7, Ex. 4 at 8, Ex. 5 at 8, Ex. 6 at 108, Ex. 7 at 121, Ex. 8 at 36, Ex. 9 at 165, Ex. 10 at 127.) The cases Defendants cite for the proposition that Dr. Kilpatrick improperly omitted sales data from his analysis are distinguishable. In Exxon Mobil v. Albright, 71 A.3d 30, 102 (Md. 2013), for example, the Court of Appeals of Maryland held the trial court should have excluded Dr. Kilpatrick’s testimony regarding assessment of diminished property values because he did not take into account “the comparable sales data of the nearly 180 real estate sales” in the area at the relevant time. See also Palisano v. Olin Corp., 2005 WL 6777561, at *5 (N.D. Cal. July 5, 2005). Here, in contrast, GAVM does consider comparable sales data.
As for the use of tax assessed values and omission of other variables, the Supreme Court has explained:
While the omission of variables from a regression analysis may render the analysis less probative than it otherwise might be, it can hardly be said, absent some other infirmity, that an analysis which accounts for the major factors must be considered unacceptable as evidence . . . . Normally, failure to include variables will affect the analysis’ probativeness, not its admissibility.
Bazemore v. Friday, 478 U.S. 385, 400 (1986) (internal citation and quotation marks omitted). Defendants’ own expert testified that tax assessed data is commonly used in the AVM industry. (Barrett Decl., Ex. 4 21-22.) Indeed, other courts have rejected challenges to Dr. Kilpatrick’s and other expert’s analyses regarding the use or omission of similar variables. See Hartle v. FirstEnegery Generation Corp., 2014 WL 1317702, at *9 (W.D. Pa. March 31, 2014) (“Kilpatrick had ‘good grounds’ for his choices in applying the hedonic regression analysis. . . . For example, even if the data relied on by the expert is imperfect, and more (or different) data might have resulted in a better or more accurate estimate in the absolute sense, it is not the district court’s role under Daubert to evaluate the correctness of facts underlying an expert’s testimony.” (internal citation and quotation marks omitted)); Federal Housing Agency v. Nomura Holding America, Inc., --- F.Supp.3d ----, 2015 WL 568788, at *11 (S.D.N.Y. Feb. 11, 2015) (“[T]he reliance by FHFA experts on tax records from 2013 and 2014 in their assessment of property valuations during the period of 2005 to 2007 may be entirely appropriate. Post-origination evidence is admissible if it tends to show the existence or non-existence of a fact during the relevant period of time. Thus, the defendants’ complaint about the use of an AVM which relied on recent tax assessed values misses the mark.”). Moreover, Dr. Kilpatrick indicated that including additional variables would have necessitated excluding large numbers of comparable properties. (Kilpatrick Decl. ¶ 23.)
Dr. Kilpatrick’s filtering and calibration techniques also do not render his analysis inadmissible. Defendants’ own expert testified that filtering techniques in a regression analysis are common, are “often reported in the literature, ” and that he uses them himself. (Barrett Decl., Ex. 5 at 126; see also Barrett Decl., Ex. 4 at 48-51.) As Dr. Kilpatrick explains, some filtering is necessary for regression models, which are especially sensitive to outliers. (Kilpatrick Decl. ¶ 43.) Moreover, the Cross-Validation Filter is not applied to the Sample Properties but only to the comparables, and it is only when the filter is entirely removed that the analysis changes significantly. (Id. ¶ 44.) While Defendants assert Dr. Kilpatrick’s filtering and calibration constitute manipulation of the data to reach a desired result, this, again, is fodder for cross-examination and not an appropriate ground for excluding his opinion.
The admissibility of Dr. Kilpatrick’s opinion regarding CAM is, in the court’s view, a closer question. Nevertheless, the court finds his analysis was sufficiently reliable for admissibility purposes, except for one minor aspect of his report. See Federal Housing Finance Agency v. Nomura Holding America, 2015 WL 353929, at *4-6 (Jan. 28, 2015) (denying the defendants’ motion to exclude Dr. Kilpatrick’s nearly identical methodology and opinions regarding GAVM and CAM, but excluding Dr. Kilpatrick’s opinions as to the subjective beliefs of the original appraisers).
First, contrary to Defendants’ assertion, Dr. Kilpatrick does provide support for his 31 questions and the weight assigned to each. He points to the USPAP standards, commonly used appraisal forms, and his own knowledge and experience in the field. (Kilpatrick Decl. ¶¶ 67-70, Ex. 24.) See Milward, 639 F.3d at 19 (“In concluding that the weight of the evidence supported the conclusion that benzene can cause APL, Dr. Smith relied on his knowledge and experience in the field of toxicology and molecular epidemiology and considered five bodies of evidence drawn from the peer-reviewed scientific literature on benzene and leukemia.”). Defendants argue many of the cited sources do not support Dr. Kilpatrick’s specific questions. And while the court struggles at times to see the connection between a given source and the CAM question, “[t]he soundness of the factual underpinnings of the expert’s analysis and the correctness of the expert’s conclusions based on that analysis are factual matters to be determined by the trier of fact.” Id. at 22 (quoting Smith v. Ford Motor Co., 215 F.3d 713, 718 (7th Cir. 2000)). Defendants’ reliance on Barletta Heavy Div., Inc. v. Travelers Ins. Co., 2013 WL 5797612, at *8-9 (D. Mass. Oct. 25, 2013), is misplaced. There, the expert failed to identify “any insurance manuals, industry publications or any other sources in support of his opinion. He has pointed to no objective sources for his opinions nor identified any definable set of insurance industry ‘best practices’ against which he measures Traveler’s conduct.” Id. at *8. In addition, the expert “offer[ed] a series of unanchored conclusions.” Id. at *9. That is not the case here.
Second, other questionnaires similar to CAM are relied upon in the market. (Kilpatrick Decl. ¶ 73.) In fact, Defendants’ expert uses such tools and attached ten commonly used appraisal review forms to his report. (Barrett Decl., Ex. 9, Ex. 16 at 185-197.) Again, unlike those commercially available tools, Defendants can replicate and test Dr. Kilpatrick’s analysis. Moreover, in a sense, the CAM is similar to a survey, which courts have found are generally admissible. See, e.g., United States v. American Express Co., 2014 WL 2879811, at *10 (E.D.N.Y. June 24, 2014) (“As a practical matter, there is no such thing as a perfect survey. . . . Accordingly, errors in a survey’s methodology . . . generally bear upon the probative weight to be afforded the survey, rather than it admissibility.” (internal citation and quotation marks omitted)).
Third, Recovco’s review somewhat corroborated CAM’s results. Although that review does not constitute independent validation of Dr. Kilpatrick’s methodology and he likely will not be able to testify about it at trial, see U.S. v. Zolot, 968 F.Supp.2d 411, 426 n.27 (D. Mass. 2013), courts may consider otherwise inadmissible evidence in the context of a Daubert challenge. See Fed.R.Evid. 104(a); Celebrity Cruises Inc. v. Essef Corp., 434 F.Supp.2d 169, 190 (S.D.N.Y. 2006) (“Accordingly, ‘in determining whether to admit scientific testimony the court may consider materials not admissible in evidence.’” (quoting Ruffin v. Shaw Indus., Inc., 149 F.3d 294, 297 (4th Cir. 1998)).
As for Defendants’ argument that Dr. Kilpatrick should not be permitted to testify as to the original appraisers’ state of mind, the court agrees. See Holmes Grp., Inc. v. RPS Prods., Inc., 2010 WL 7867756, at *5 (D. Mass. June 25, 2010) (“An expert witness may not testify as to another person’s intent. No level of experience or expertise will make an expert a mind-reader.”). Judge Cote’s analysis of this issue in Federal Housing Finance Agency v. Nomura Holding America, 2015 WL 353929, which involved similar passages from the report Dr. Kilpatrick submitted in that case, is particularly on point:
FHFA will apparently attempt to establish that at least some appraisers did not actually believe at the time that they submitted their appraisals that the appraisals accurately reflected the property values. But, while it may be able to accomplish this task through circumstantial evidence, it may not do so by proffering an opinion of its expert on the subjective beliefs of the appraisers.
Accordingly, passages such as the following are stricken from the Report, and FHFA may not elicit equivalent testimony from Kilpatrick during his direct testimony:
“I conclude that it is highly doubtful whether a reasonable appraiser adhering to the appraisal standards and practices applicable during the relevant time period could have concluded that the original appraised values . . . .”
“I conclude that it was highly questionable whether a reasonable appraiser could have believed that the appraisal values . . . were credible . . . .”
“[I]n my opinion, no reasonable appraiser adhering to appraisal standards applicable at the time could believe the appraisal credible.” “[N]o reasonable appraiser adhering to the appraisal standards and practices applicable at the time could have believed the appraisals were credible.”
FHFA argues that Kilpatrick is simply offering classic standard-of-care testimony about the degree to which certain Nomura Appraisals deviated from USPAP and applicable appraisal practice, and from which a fact finder may reasonably infer that an appraiser did not believe that the appraisal was accurate at the time it was made. FHFA is correct that Kilpatrick may opine about the degree to which the historically performed appraisals deviated from the standards established by USPAP. What the expert may not do, however, is take that analysis further and fashion an opinion about the appraisers’ subjective state of mind. As Kilpatrick acknowledged during his deposition, he is not in a position to opine on the state of mind of any of the appraisers.
Id. at *5-6. Judge Cote, however, permitted Dr. Kilpatrick to opine as to the appraisals’ “credibility, ” a defined USPAP term which “reflects an objective assessment of an appraisal.” Id. at *6. The court reaches the same conclusion here. Accordingly, Dr. Kilpatrick may not directly testify as to the original appraisers’ state of mind, but he may testify as to the appraisals’ “credibility.”
B. Motion to Exclude Mr. Butler’s Testimony
Defendants seek to exclude Mr. Butler’s expert testimony on the grounds that he improperly relied on the Minimum Industry Standards, used the wrong guidelines at times, based many findings on missing documents, used post-origination information, and relied wholesale on Dr. Kilpatrick’s analysis. Plaintiff, again, asserts Defendants’ challenges are meritless and, at most, go to the weight, not the admissibility, of Mr. Butler’s opinions.
Defendants argue forcefully that Mr. Butler cannot base his material misrepresentation findings on the Minimum Industry Standards because it is only the failure to comply with the specific underwriting guidelines described in the offering documents which is relevant. Plaintiff argues, on the other hand, the Minimum Industry Standards are relevant when the applicable guidelines cannot be found or are silent on important issues. Plaintiff may be correct that the Minimum Industry Standards have some relevance under certain circumstances, but the court will defer deciding the issue at this time. For now, the court concludes that Mr. Butler’s preparation of the Minimum Industry Standards was sufficiently reliable because they are based on specific standards and experts may opine as to industry customs and practices, if sufficiently qualified. See Federal Housing Finance Agency v. Nomura, 2015 WL 930276, at *4 (S.D.N.Y. Jan. 29, 2015) (“FHFA has shown that the process that [the expert] employed to identify the minimum underwriting standards that prevailed in the industry during the relevant timeframe is sufficiently reliable to make his testimony admissible. It is not novel for an expert to opine on customs and practices within an industry.”); Assured Guarn. Mun. Corp., 920 F.Supp.2d at 505 (“[E]xperts qualified by their experience may testify to their conclusions as long as they exhibit ‘in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field.’ . . . Walzak articulated the sources of information on which she relied . . . and applied her experience in the underwriting industry to the resultant findings to make a determination as to whether the deficiencies in a particular loan rose to the level of materiality.” (quoting Kumho Tire, 526 U.S. at 152)). If the court precludes Mr. Butler from testifying about Minimum Industry Standards, it will not be because he is insufficiently qualified or prepared them in an unreliable manner but, rather, because those standards are not relevant to the task at hand. See Federal Housing Finance Agency v. Nomura, 2015 WL 930276, at *3-4.
Defendants next argue Mr. Butler’s opinion should be excluded because he applied the wrong guidelines at times. As an initial matter, the court will not take Plaintiff’s invitation to preclude this argument based on Defendants’ alleged discovery violation of failing to identify the applicable guidelines in response to an interrogatory, although it does take into consideration the seeming unfairness of refusing to provide this information but then faulting Plaintiff for applying the wrong guidelines. Plaintiff, the court finds, waited too long to raise this issue, despite its awareness of Defendants’ refusal to provide the discovery for some time. See Monteagudo v. Asociacion de Empleados del Estado Libre Asociado de Puerto Rico, 554 F.3d 164, 172 n.8 (1st Cir. 2009) (“[T]his was a discovery issue that AEELA should have addressed earlier either by way of a motion to compel or a request for sanctions under Fed.R.Civ.P 37(c).”). In any event, only a small portion of the loans were reunderwritten using the purported wrong guidelines, and Mr. Butler has since explained that his analysis does not materially change even when using the guidelines to which Defendants point. (Bulter Decl. ¶¶ 23-25.) Any error on Mr. Butler’s part does not render his analysis inadmissible. See In re Washington Mut. Mortg. Backed Sec. Litig., 2012 WL 2995046, at *6 (W.D.Wash. July 23, 2012) (“Defendants have raised some question as to whether Holt properly applied each of the guidelines. But this hardly requires exclusion of his report, particularly where both sides seem to agree that application of the underwriting guidelines requires some discretionary decision-making. It is up to a jury to determine whether the purported flaw’s in Holt’s analysis renders his opinion unworthy of merit.”).
Defendants also fault Mr. Butler for basing many of his findings on documents missing from the loan files, contending that this does not demonstrate non-compliance with guidelines at the time of the original underwriting because there could be a variety of reasons why those documents are missing now. The court concludes missing loan documents at this time may provide circumstantial evidence that those documents were also missing at the relevant time. Ultimately, the trier of fact will have to make that determination. But it is not a reason to exclude Mr. Butler’s opinion. See Milward, 639 F.3d at 22 (“When the factual underpinning of an expert’s opinion is weak, it is a matter affecting the weight and credibility of the testimony-a question to be resolved by the jury.” (quoting U.S. v. Vargas, 471 F.3d 255, 264 (1st Cir. 2006)).
Defendants’ arguments regarding Mr. Butler’s use of post-origination information are similarly unavailing. As Plaintiff asserts, the guidelines generally required underwriters to verify the reasonableness of information and to follow up on red flags. Thus, post-origination sources may provide circumstantial evidence of failures in that regard, as many courts have held. See, e.g., Federal Housing Agency v. Nomura Holding America, Inc., 2015 WL 568788, at *10 (“The defendants are wrong [that post-origination information is irrelevant]. Direct or circumstantial evidence, regardless of when that evidence first became available, would be relevant if it helped to demonstrate that an Originator did or did not follow its own underwriting guidelines or that the loan did or did not qualify under the Originator’s guidelines.”). Defendants’ specific criticism of the use of Bureau of Labor Statistics (“BLS”) data likewise fails. Mr. Butler referenced BLS data because other sources do not maintain historical data but BLS does. Moreover, similar to the expert in Assured Guar. Mun. Corp., 920 F.Supp.2d at 506, Mr. Butler adequately accounted for the potential that BLS data might understate incomes by finding a borrower’s stated income unreasonable only when it exceeded the 90th percentile of the BLS index for the borrower’s occupation, position, and geographic location. See also Cummings v. Standard Register Co., 265 F.3d 56, 64 (1st Cir. 2001) (“Duffy offered sufficient explanations for why he chose to use BLS data and Cummings’ 1997 salary in his calculations.”).
Lastly, the court finds no issue with Mr. Butler’s reliance on Dr. Kilpatrick’s data for his conclusion that loans violated the guidelines’ LTV requirements. Mr. Butler did not simply parrot or rehash Dr. Kilpatrick’s finings but, rather, reached independent conclusions as to breaches of underwriting guidelines and whether those breaches substantially increased the loans’ credit risk. See Ferrara & DiMercuio v. St. Paul Mercury Ins. Co., 240 F.3d 1, 9 (1st Cir. 2001) (“[T]he opinion he rendered was his own . . . . Federal Rule of Evidence 703 allows Malcolm to have taken O’Donnell’s report and opinion into account when forming his own expert opinion.”) Moreover, as the First Circuit has explained, “when an expert relies on the opinion of another, such reliance goes to the weight, not to the admissibility of the expert’s opinion.” Id.
For these reasons, the court ALLOWS Defendants’ motion to exclude the expert testimony of Dr. Kilpatrick insofar as it seeks to preclude him from testifying as to the original appraisers’ state of mind but otherwise DENIES the motion. (Dkt. No. 355.) The court also DENIES Defendants’ motion to exclude the expert testimony of Mr. Butler. (Dkt. No. 359.)
It is So Ordered.