Data Warehouse/Dashboard: Supermarket Model
Most school systems have numerous disjointed data platforms with various combinations adopted by leaders and stakeholders looking to solve their own data problems. While meeting individual unique data needs, multiple data platforms—often layered on top of each other in isolation—have hampered educators’ efficient use of data.
For educators, accessing student data often involves logging into and navigating multiple websites. This inconvenient and sometimes painfully inefficient process frustrates educators and can discourage them from using data to guide and inform decisions.
In response to this siloed data model, the EdTech industry developed various data warehouse and dashboard products that gather data from multiple platforms, providing educators with “one-stop shopping” to meet their data needs.
At the basic level, data warehouses and dashboards allow educators to see raw student or aggregated data in a tabular or graphical format. In recent years, data warehouse and dashboard vendors have increasingly integrated machine learning and artificial intelligence into their products and can now visually present complex data in more intuitive ways, with some even providing predictive warning signs.
Design-wise, most data warehouse and dashboard products resemble supermarkets where the layout and shelving are organized into categories of goods, such as food, clothes, pharmacy, and electronics. These goods are further separated into sections and by shelves in each area on the basis of their category or function.
Similarly, the contents of many data warehouses and dashboards are organized into categories of data. For instance, users typically access academic achievement data and behavior data on two different sets of web pages. To view student-level data, they might have to drill down one level and then continue to drill even further. Just as we can easily spend over an hour in a Walmart or Costco looking for all the items we need, we can also spend considerable time navigating the layers within a data warehouse or dashboard site to find the data we need.
This supermarket warehousing design is a clean, clear structure that helps the user navigate often-complex websites that provide access to hundreds of data points. When it comes to the practical data uses of many educators, however, all that rich data are still siloed even though they can be accessed in “one place.”
It is not uncommon for educators to visit a warehouse or dashboard website, download data from multiple pages, then manually combine them on a separately created spreadsheet. Despite the inefficiency associated with this approach, these customized spreadsheets allow educators a tailored view of pertinent information. For instance, instead of navigating back and forth between multiple web pages, they may now view all of a student’s information in one row.
These extra steps tell us that we have not solved educators’ data problems. Rather, we have managed to make their data problems somewhat less painful.
Ad Hoc Report: Agent Model
Many districts provide designated personnel to answer data requests from educators. Such requests may involve idiosyncratic data points that are unavailable on the district’s data warehouse or dashboard websites. Many times, support personnel bridge the gap between data availability and usability. That step is necessary because educators do not have the skills to download the data themselves or because they believe their time is otherwise best spent while someone else does the research.
In essence, district staff members who answer data requests are agents whose responsibilities are to retrieve and transform data into something that educators can use. Behind this model is a user-centered approach that can be described as “tell us what you need, and we will get it for you.”
The main strength of this model is that data outputs can be customized to meet the specific needs of educators so they can immediately use the final product. This not only saves time for educators, but also addresses the equity issue for educators who do not have the skill sets to capitalize on the rich data made available to them through a data warehouse or dashboard.
The problem with the agent model of data support is scalability. Many districts—especially smaller ones without the resources to hire additional staff to focus on data support—may stretch their already-lean information technology departments or they may ask personnel who already wear multiple hats to take on yet another task.
To make matters worse, many data requests are repetitive and tend to occur during the same period. For example, all schools and central offices need data to develop their improvement plan during the planning season. For staff members who receive and answer these data requests, repeating the work for each department and school is tedious and not the best use of their time.
App: Online-Shopping Model
Undoubtedly, these two-dimensional supports have allowed educators to use data to improve their decisions and practices. Data warehouses and dashboards provide one-stop shopping for educators; they are comprehensive, responsive, and scalable. On the other hand, ad hoc reports offer educators personalized and flexible data points in a ready-to-use format.
Further growth and use of both models are not only necessary but vital in our efforts to better support educators’ data-driven decision making. That said, the two approaches are inadequate for supporting data use, which is the focal point of data support.
We have learned from interacting with teachers and administrators that over time, they have developed a number of routine data uses, each involving a set of data points presented in specific ways. For example, to develop school improvement plans, school leadership teams usually compile comprehensive summative assessment data, including end-of-year attendance, behavior, culture and climate, and state assessment data aggregated at the school level.
School intervention teams, in contrast, rely on student-level screening data collected from multiple screening instruments at the beginning of a school year to decide which students should receive what intervention; they then regularly collect monitoring data, such as attendance, behavior, intervention, and formative assessment on a weekly, biweekly, or monthly basis to inform adjustments.
In most cases, data points for these routine data uses remain unchanged until a new assessment or screening tool is adopted. Data formats and presentations tend to be stable as well. The primary task related to each use is to update the spreadsheets, graphs, or reports with the latest data.
As far as those routine uses are concerned, data-centered warehouses and dashboards provide most, if not all, of the data points educators need; however, they often require extra steps from educators to make them usable. User-centered ad hoc reports give educators data they can use directly but lack scalability and responsiveness.
Our district’s solution for better supporting routine data uses is an app with a use-centered design. This app is accessible from both a web browser and a mobile device.
On the report drop-down, each report addresses a routine data use (or a class of routine data uses) involving different combinations of the same set of data points. To request a report, a user simply selects the report from the drop-down list, specifies data points, and clicks on Submit. The user receives an email notification with a link to the data report in the predefined format, which can be used directly for instructional or improvement purposes.
Using the app to readily obtain data for use is akin to online shopping. That is, users ask for what they need and get the final product delivered quickly in the format they can use immediately.
With this app, our elementary school administrators, with a few clicks, can ask for and receive within minutes the data that are ready to use. Principals can submit requests on their own at any time and always receive the most recent data. The process not only saves time, but also improves equity since the app is so easy to use that any educator can complete a request with minimal or no training.
Three-Dimensional Data Support
With data apps complementing a warehouse or dashboard and ad hoc reports, we can now provide three-dimensional data support that covers different stakeholders’ varied needs and skill sets to facilitate and promote data use. In this new data support ecosystem, most routine data requirements can be addressed by various use-centered apps. Ad hoc reports can handle novel data uses or uses involving data that are unavailable on a district’s data warehouse or dashboard site.
Data warehouses and dashboards continue to empower educators who have an innovative bent as well as the necessary skills to be creative in developing new uses with the rich data that are available.
In addition to supporting educators’ data needs with their own strengths, the three models can further enhance this three-dimensional support when integrated. For example, a once-novel data use supported by the ad hoc report model might become popular or might be deemed by the district to be important for broader implementation. In that case, it can be added to the already-developed app as a new drop-down option for wider adoption and use. As a result, support for that data use is scaled up to cover all potential users with ready-to-use data on demand instead of waiting for support staff to respond.
This efficient use of resources also means that support staff are freed to provide better ad hoc report support rather than doing the same tasks repeatedly for different people and at different times.
Although vendors have primarily adopted a data-centered design when building data warehouses and dashboards, no formidable technology barrier prevents them from adding use-centered features to their products. Specifically, they can include a page that is designed for use-centered reports on their products. Once a report is selected, it can be displayed on the site or downloaded to a user’s computer. In this way, their products provide a one-stop shop that provides both the supermarket and online shopping experiences. How convenient and powerful that would be for educators as data consumers!
In the past, we have largely treated all data users and their needs the same. Differentiating between users and focusing on use allow us to segment those diverse needs and take advantage of each model to provide more efficient, effective, and equitable data support to educators. As a result, educators can focus on leveraging instead of generating the power of data to help students succeed.