June 27, 2024

In the section below is a response from Dr. Jardine, Chief Data Officer for the Cook County Assessor’s Office to my attempts to communicate concerns—using data—about CCAO’s over-assessment of old housing stock homes in Chicago’s gentrifying neighborhoods. The response came via Scott Smith, who is Chief of Staff for the Assessor’s Office.

For context on this exchange with the Assessor’s Office and links to my work, see this page.

I have communicated in various ways with the Assessor’s Office. I am not sure which communication Dr. Jardine is responding to, that is, what she has or has not read of my work. For example, she says that I do not describe my methods and yet I include details such as this:

1.     How much have assessed values changed for longtime homeowners in areas of the city seeing sharp property tax increases?

To answer this question, I used a sample of 30 small old homes purchased between 1999 and 2008 that have not sold since—as of mid-2022, ownership of these homes had not changed. This sample is from a much wider area than covered by my first study, and the homes are more diverse, though all are in class 2-02. Using Cook County’s parcel sales data for 1999 to mid-2022, I identified the homes using “2-02” classification and the area “13-23.” “2-02” homes are one story single-family homes under 1000 square feet. “13-23” refers to the first four digits of a Property Identification Number; it is a one square mile area within Jefferson Township from Belmont Road to Irving Park Road and from Kedzie Avenue to Pulaski Avenue. My dataset query initially generated a total of 63 “2-02” sales. Most of the loss in sample size is due to homes being reclassified as larger “2-03” or other kinds of homes, presumably after rehab work. This loss is consistent with what we know about “starter homes.” They are becoming extinct.

This sample is intended to represent a much larger group of longtime Chicago homeowners, including those who purchased their homes before the year 2000. In my initial study, 12 of 20 homeowners fell into this group.

The change in assessed value for the 30 homes from 2018 to 2021 is 23.7%. This is virtually the same as the 23.5% change I found in my immediate neighborhood for a combination of twelve “2-02” homes and eight “2-03” homes. Assessed values are shown in Table 1 in the next section.

Analysis of the data tells us that longtime homeowners of old Chicago homes in this area of the city have seen an 24% change overall in assessed value between 2018 and 2021 for their properties. (My analysis goes on to show that longtime homeowners experienced a change in assessed value that is four times greater than homeowners of luxury homes. Again, I detail how I identified luxury homes. My analysis from there goes on to show that the assessment increases are not justified by sales prices and that the evidence the Assessor’s Office provides to show that assessment increases are due to sales price increases is weak.)

Summarizing my comments (see section below) to Dr. Jardine’s response:

Dr. Jardine has not provided a thoughtful response to my concerns. The biggest point she is missing is that I am a homeowner who worked hard—using information and data provided by the Assessor’s Office—to understand my home’s assessment (Chicago 2021) and why it resulted in a very large tax increase. I am not a single case. Many of my neighbors saw similarly high increases in their assessments and taxes. Dr. Jardine chooses not to engage with my understanding.

The Assessor’s Office explanations to homeowners about their assessments are general and insufficient, and as a result, not informative. Furthermore, they have become, over time, stripped of essential information for homeowners to know, for example, that a tax increase for one group of homeowners means another group has experienced less of an increase or even a decrease. (Compare the 2024 valuation report for Rogers Park with the 2021 report to see these changes.) The Assessor’s Office seems to be operating on the attitude, “The less information homeowners have the better.”

The concerns of the Assessor’s Office and my concerns are aligned in recognizing that its data challenges affect its assessments. We diverge on the issue of how systematic these assessment problems are, why and whom they effect. The Assessor’s Office uses sales ratio studies that provide limited statistics for targeting equity problems in assessment. The limitations of these statistics are noted by expert Dr. Carmen Quintos. Dr. Quintos work is referenced in the exchange below. I sent Dr. Quintos peer-reviewed paper to the Assessor’s Office. Dr. Jardine seems not to have read it despite the fact that her staff members are enthusiastic about it. The work I did using data for my immediate neighborhood suggests the CCAO’s data problems lead to systematic assessment problems. Quintos’ work is important for understanding the “ifs” and “whys” of these problems within and across Chicago’s townships from a statistical perspective. The Assessor’s Office seems to be saying its sales ratio studies indicate its assessment work is completely sound. For a scientific perspective, this is not true.

An essential point for homeowners, albeit one the Assessor’s Office has never made, but which lies behind this exchange, is that a “fair” assessment in Chicago is not one AT market value, but rather it is one in the same proportion to market value as homes in mostly white and wealthy neighborhoods of Chicago. If homes in mostly white and wealthy neighborhoods tend to be assessed at 85% of market value, so too should homes in Chicago’s other neighborhoods. The Assessor’s Office academic partner at the University of Chicago makes a similar point, though does not draw attention to the racial and class implications as I do.

If I have further communications with the Assessor’s Office, it will be around new data, for example, the 2024 assessments for Chicago. Unfortunately, the CCAO’s office is providing less information in its assessment reports (compared to 2021) with which to understand assessments.

Below is the email message from Scott Smith, CCAO Chief of Staff:

Dr. Walker:

Please see below for a response to your correspondence on October 30th. The response is written by Dr. Nicole Jardine, Ph.D. She is our Chief Data Officer.

Her background can be viewed here:

https://www.linkedin.com/in/nicole-jardine-98a6b521/

I’ve cc’d Dr. Jardine on this email in case you have any follow-up questions for her.

Scott

Dr. Nicole Jardine’s Response and my comments in italics

I was very excited to read that Dr. Walker, in her letter about Assessor’s Office (AO) reassessments, is focused on assessment equity. It’s the same issue that motivates the work of the entire Data team and many of the Office’s public servants. 

It’s difficult to speak to the individual properties without knowing the PIN numbers of the properties referenced. It’s also difficult to appreciate how systemic these issues are or how these particular sales were selected from just the letter, which does not have a methods section. If there exists a list of PINs or a detailed methodology used to select these PINs I could better comment.  

Furthermore, it’s also difficult to know which PINs have had changes to its characteristics or its assessment (and whether assessment changes were made by the AO or the Board of Review) since Chicago’s assessment in 2021.  

Comment: If this is all she has to say, Dr. Jardine is indicating that she is not interested or curious about what I have found as a homeowner who has explored the data. I did not do this work as an academic researcher, but rather as a homeowner trying to learn about why my assessment, and many of my neighbors’ assessments, have increased dramatically since Fritz Kaegi took office. In order to understand my assessment, I had to learn about how assessments work, including appeals. It was a steep learning curve. My data work got tighter/more focused as I learned about the data.

I am completely confident in the patterns I found for the 2021 Chicago assessment for the geographic areas I have studied, that is, my gentrifying community. To repeat my findings, which I am sure were communicated to Dr. Jardine: assessments for modest old homes are too high (the homes are over-assessed), assessments are more accurate and consistent for recent new construction homes, and they are inconsistent and often inaccurate for rehabs. Do I think that the analysis could have been tighter? Yes. Do I think a tighter analysis would have changed the overall findings? No.

I also found that the above patterns are related: new construction and rehabs drive up assessments for modest old homes much more than they drive up actual home prices for the same homes (modest old homes require significant investments to improve to meet gentrified standards, which home prices reflect.)

Due to these patterns, our taxes increased by about $2000 in 2021. Many neighbors of modest old homes like ours saw similar increases. I know from newspaper accounts that many other homeowners in Chicago, particularly Latino neighborhoods, experienced similar increases.

The Assessor’s Office may not be interested in these patterns (and resulting taxes), but if that’s the case, Dr. Jardine should say so. She should not imply that I, with my limited tools and time, should do a better job. My taxes pay the salaries of Assessor’s Office employees so they can do the heavy lifting to provide fair assessments. I thought that an Office concerned about fairness and equity would be interested in what I learned and might consider following up. Instead I get explanations that communicate, “It is what it is. We are doing a good, actually a great, job. Homeowners should accept their sharply higher assessments and taxes.”

Dr. Jardine continues:

Instead, I’ll focus on the broader systemic issues that drive us and motivates our work: price-related equity.  

In Table 1, Dr. Walker presents a table that re-summarizes the AO’s self-published ratio studies. One technical note is that the studies from 2021 should be taken with a grain of salt because they reflect an early stage of the property assessment cycle. It’s easy to miss, but the report specifies that these ratio studies are at the “modeling stage.” Although the model does produce initial estimates of home values, these estimates are then refined by analysts, property by property and neighborhood by neighborhood, before they are mailed. As such, the ratio studies used to produce Table 1 might have been different had they been conducted on the mailed assessments (instead of the model stage). 

Comment: Dr. Jardine is explaining away the data (text in yellow). Rule number one for a public servant being held accountable by data is: If you don’t like what the data reveals, find a way to explain it away. First, she dismissed my findings as not worthy of her attention. Now she is explaining away data that appears in the Assessor’s Office own reports. Data can always be better. The more important question is whether better data would change overall patterns. The data she is explaining away reveals a bias in assessments across Chicago that favors the wealthiest and whitest communities. Of course, she says, “take it with a grain of salt.” One implication of this bias is that where assessments were lowered in Chicago, which largely benefited Chicago’s Black communities, they were probably not lowered enough.

Dr. Jardine continues:

In addition, I’d like to highlight the ways that the AO goes above and beyond here. 
1.     Assessors in Illinois are not required by law to conduct ratio studies on itself, nor to publish them, but the AO is committed to doing so. (Other states are required to do so.) We publish the IAAO’s “usual” statistics for measuring assessment accuracy (median ratio), uniformity (COD), and equity (PRB and PRD). 

2.     Furthermore, we decided to not just publish median ratios, but also the ratios for the top and bottom terciles of the price spectrum. Although this analysis is uncommon, we choose to add it because it is clearer and more immediately understandable than the PRB or PRD. To us, transparency means not just publishing data, but publishing data that is understandable.  

3.     Finally, these reports are not buried in multi-page documents or tables of text. They are a predominant part of the township reporting, are referenced in press releases, and are easily findable from the Assessor’s Website.   

Comment: I thank the Assessor’s Office for the information it uses to document the 2021 Chicago Assessment. However, it stripped away a lot of useful information in subsequent assessment reports. For example, compare the reports for the 2023 Oak Park Assessment with the 2021 Jefferson Township Assessment. In providing less information, like ratios for the top and bottom terciles of the price spectrum, the Assessor’s work becomes less transparent and understandable to someone like myself. Notably, recent reports do not provide information about how the Assessor’s Office divides homes into valuation categories within a given area to produce assessments. To understand one’s assessment using CCAO information, one needs not only to know the median assessment value, but the values that separate homes into three price categories. This enables a home owner to identify the price category for her home and consider whether or not it seems correct.


I was a little confused by the reference to the Quintos paper’s suggestion about analyzing subgroup effects and price-level groupings. That’s exactly what we’ve done with analyzing and publishing ratios for the top third, middle third, and bottom third price groupings. (I’ll also note that the IAAO does mention visualizing the distribution of ratios in its section 5 of the Standard on Ratio Studies, but warns that there are boundary effects in comparing the highest and lowest price strata in section 5.6.)  

Comment: It shouldn’t be confusing to Dr. Jardine for me to point out that the statistics the Assessor’s Office uses is not informing them sufficiently about problems in their model. Quintos’ paper starts with the premise that sales ratios are not sufficient for understanding inequities in assessments. The Assessor’s Office currently relies solely on sales ratios to assess the accuracy of its work (see Dr. Jardin’s point #1 above).


We’re excited that people are actually reading these ratio studies and interpreting them. Insofar as actual assessment equity: the studies show that compared to 2018, overall, assessment equity has improved (though this is not summarized in Table 1). There’s always room to do better and we are focused on just that. I, personally, am proud to work for an office that supports publishing self-studies that indicate we are not yet perfect, and does so in a way that is clear and transparent.  

Comment: This kind of cheerleading worries me quite a lot. A lot of people have been overtaxed because of problems with the assessment model. These problems are not revealed by the statistics the Assessor’s Office currently uses to evaluate its work. It appears from Dr. Jardin’s response that the Assessor’s Office has decided to not see/probe/understand these problems.

The Assessor’s Office information is NOT clear and transparent to homeowners, who ask the question, why has my assessment gone up so much and what will it mean for my taxes. Clearness eventually comes with the tax bill. The Assessor’s Office needs to provide a non-technical explanation in clear language that homeowners can understand. I’d be happy to draft one based on what I’ve learned. It could then be shared in focus groups of homeowners to learn from them how to improve it.


Dr. Walker referenced Carmela Quintos’s paper on Gini coefficients. AO leadership are also fans of her work and are in regular correspondence with her. Dr. Walker might be excited to know that we added Gini metrics to our ratio study analysis package this summer (https://github.com/ccao-data/assessr/issues/3), and we are currently updating our codebase so that the Gini metrics, it in addition to other methods and metrics to analyze assessment equity, can be used for the 2024 Chicago Reassessment.  

Finally, another reference made is about factor effects and locational effects in linear regression. We incorporate many location factors (predictors) in the model. This is important because so much of real estate value is captured by location factors. https://github.com/ccao-data/model-res-avm#features-used  

Comment: It sounds as if the Assessor’s Office is using Quintos’ work selectively. It also sounds as if Dr. Jardine hasn’t read (or perhaps thought deeply about) Dr. Quintos’ work and so doesn’t understand its potentially far-reaching implications for analyzing Chicago’s assessment model.

Chicago’s next citywide revaluation will be in 2024 and we have multiple initiatives to improve both the data and the model.  

Regarding data improvements: 

·        One of the questions asked was, “how do you know that improving data on home characteristics will address the assessment problems that CCAO’s statistics are flagging?” There’s a paper to that effect published by one of our other large-jurisdiction colleagues: 
Jennifer, R. (2021). Garbage in, garbage out: Implications of data quality for valuation models. Journal of Property Tax Assessment & Administration, 18(1). Retrieved from https://researchexchange.iaao.org/jptaa/vol18/iss1/1 

Comment: This goes back to my point about Dr. Jardine not understanding Dr. Quintos’ work.

       Another question was, “are homeowner appeals the only recourse for correcting assessments in the meantime?” Happily, no. Our Data Integrity team relies on data sources to detect large changes in property characteristics. We even have a questionnaire form by which property owners can voluntarily send updated property characteristics, and used it in a recent campaign to support accurate assessments of small apartment buildings: https://www.cookcountyassessor.com/news/what-do-if-you-receive-postcard-mail-assessors-office  

Comment: It sounds that for homeowners like myself, the answer is “yes.”


Regarding improvements to the model, we’re doing a lot of work in preparation to model the City. Much of it is publicly trackable here: https://github.com/ccao-data/data-architecture/issues. I’ll highlight some personal favorites:  

·        We’re working on automatically ingesting Chicago permit data, so that our Data Integrity team can more efficiently update property characteristics. This would help resolve issues mentioned in the letter about rehabs and new construction. Track our work here: https://github.com/ccao-data/data-architecture/issues/158  

·         We’ve developed an entire codebase to try to detect sales that should not be used to train the model. As an example, one process is to detect likely “flips” – properties that are purchased, rehabbed, and re-sold. https://github.com/ccao-data/model-sales-val  

Comment: Good. Will this work be reflected in the 2024 assessments for Chicago?


Finally, the letter’s conclusion mentions that “the mass appraisal approach is not working” and proposes in favor that their neighbors use “market values for similar homes nearby.” The point of the mass appraisal model is that it learns from arms-length transactions of similar homes nearby to fairly and accurately estimate the values of unsold homes. Doing this well requires having accurate home data for sold and unsold homes, as well as a model that learns from that data. Our goal is, as always, to have an accurate valuation so that homeowners don’t need to take any action to correct their property’s assessment.  

Comment: More cheerleading. Dr. Jardine is not writing this for me. I already know that the mass appraisal approach does not work for homes like mine (it inflates valuations for modest old homes) and that I must rely on the appeal process where I can present market values for similar homes nearby. Dr. Jardine has chosen to not engage with the gist of my work, but rather is cheerleading for the mass appraisal model used by the Assessor’s Office.


I thank you for forwarding me this letter and would like to end with a few additional links and comments that might be helpful: 

·        We’ve published our list of tests we hope to tackle for the 2024 model here: https://github.com/ccao-data/model-res-avm/issues  

·        Property taxes on a Chicago home are affected by assessments of every other property in Chicago, including large commercial properties. Commercial assessments can have extremely large effects on a homeowner’s bill. We have lots of guides and reporting here: https://github.com/ccao-data/wiki/blob/master/Handbook/Resources.md#property-tax-guides-and-reporting

Comment: Of course Dr. Jardine ends with this point. Fritz Kaegi seems much more focused on commercial property assessments and influencing the Board of Review so his assessments will not be reduced than making sure he is doing the best possible job to provide residential homeowners with fair assessments.

FINAL REFLECTIONS

I have puzzled over the question of how an agency can/should respond to a systematic problem of its own making, particularly when the agency is led by an elected official. My answer is that the Assessor’s Office needs to fulfill its promise of transparency to clearly and concretely explain to homeowners how their homes are assessed, the ways in which its data challenges may lead to over-assessment, and why the appeals process is currently the best option for addressing their concerns of over-assessment. While the Assessor’s Office prides itself on transparency, its transparency is largely technical in nature. It does not explain to homeowners in clear, direct and specific language what its work means for their assessments.

As a voter, I understand that an elected official intent on serious and needed reforms may not get everything right from the get-go. Still, I want the elected official to be straight with me about how his/her agency’s challenges affect determinations that affect my family and my options for obtaining a fairer determination when the reforms fall short. Assessor Office communications that acknowledge and at the same time, explain away challenges/problems contribute to the obscurity and inaccessibility of the system, as perceived by homeowners. The obscurity protects the Assessor’s Office, not homeowners.

Dr. Jardine’s response is determinedly agency-centric. She is not responding to me as a homeowner with legitimate concerns about my home’s assessment. I am much less concerned about the wonderful work the Assessor’s Office is doing (process) than about the fairness of the assessments it produces (results). While Fritz Kaegi would consider a $2000 tax increase to be high, according to this account, he would likely place the blame on the Board of Review and its lowering of assessments for commercial properties. I recognize the Assessor is protecting his work from criticism that might derail his efforts. However, having worked for a leadership program in education, I know all of this comes down to leadership. Fritz Kaegi may want to reform the assessment system, but it appears he is not willing to do a thorough investigation of the equity of his assessments, for example, by fully applying Quintos’ statistical concepts and through small-scale indepth studies that can provide him with deeper understandings of assessments at the neighborhood level.

The source of my frustration with the Assessor’s Office is that I expect that the Assessor should be accountable for systematic over-assessment. But I have learned that I as homeowner am accountable for the fairness of my home’s assessment. If I think my home’s assessment is too high, I can appeal to the Assessor’s Office, to the Board of Review, and at the state level. Because the Assessor is operating in good faith to reform the system, he is off the hook with regard to the bigger problem of systemic over-assessment (unless low- and moderate-income homeowners, like myself, become so frustrated by financially-punishing assessments that they vote for someone else.) When all is said and done, I take my accountability for my home’s assessment seriously and will continue to analyze data for my neighborhood to understand whether the home assessments I receive from Cook County are fair.