How to Use Scribbler for Time Series Analysis in Data Science Projects

Scribbler is an excellent tool for performing time series analysis in data science projects. You can load, analyze, and visualize time series data directly in your browser and you can apply advanced statistical models like ARIMA for forecasting.

Blog> Categories: Scribbler, DataScience

Table of Contents

Scribbler is a powerful notebook tool that allows you to run JavaScript and HTML code in the browser, making it an excellent platform for time series analysis in data science projects. In this article, we’ll explore how to use Scribbler to perform time series analysis, including loading data from an external source and conducting basic analysis.

Getting Started with Scribbler #

Scribbler notebooks consist of cells that can contain either HTML or JavaScript code. You can also load external libraries using the scrib.loadScript function, which enables you to leverage various JavaScript libraries for data analysis and visualization. First, let’s create html cell to hold data and chart:

HTML Cell

  <div id="chart"></div>

Loading Data from an External Source #

Now let’s load time series data from an external source. We’ll use the Yahoo Finance API to fetch historical stock prices for a given company. In the code below we are using All Origins for CORS security.

JavaScript Cell

async function loadYahooFinanceData(symbol, startDate, endDate) {
      const url = `https://query1.finance.yahoo.com/v7/finance/download/${symbol}?period1=${Math.floor(new Date(startDate).getTime() / 1000)}&period2=${Math.floor(new Date(endDate).getTime() / 1000)}&interval=1d&events=history&includeAdjustedClose=true`;
	
      const response = await fetch(`https://api.allorigins.win/get?url=${encodeURIComponent(url)}`);
      const responseJSON = await response.json();
	  const dataURL=responseJSON.contents;
	  const base64String = dataURL.split(',')[1];
     const csvText = atob(base64String);

	  console.log(dataURL);
      return csvText;
    }

    function parseCSVData(csvData) {
      const lines = csvData.split('\n');
      const headers = lines[0].split(',');
      const data = lines.slice(1).map(line => {
        const values = line.split(',');
        const entry = {};
        headers.forEach((header, index) => {
          entry[header] = values[index];
        });
        return entry;
      });
      return data;
    }
var timeSeriesData

//Load the data
(async function(){
  const symbol = 'AAPL'; // Apple Inc.
  const startDate = '2023-01-01';
  const endDate = '2024-01-01';
  const csvData = await loadYahooFinanceData(symbol, startDate, endDate);
   timeSeriesData = parseCSVData(csvData);
  scrib.show("loaded data");
})();


Time Series Analysis #

Once we have the data, we can perform basic time series analysis. In this example, we will calculate the moving average, a common technique used to smooth out short-term fluctuations and highlight longer-term trends.

    

    function calculateMovingAverage(data, period) {
      return data.map((entry, index, arr) => {
        if (index < period - 1) {
          return { ...entry, movingAverage: null };
        }
        const slice = arr.slice(index - period + 1, index + 1);
        const sum = slice.reduce((acc, val) => acc + parseFloat(val['Close']), 0);
        const movingAverage = sum / period;
        return { ...entry, movingAverage };
      });
    }

    

Visualizing the Data #

For visualizing the data, we can use Plotly, a powerful charting library. Plotly already comes pre-loaded in Scribbler. Let’s add a simple line chart to visualize the closing prices and the moving average.

Code Cell

let  movingAverageData = calculateMovingAverage(timeSeriesData, 5);
let closingData=timeSeriesData.map(entry=>[entry['Date'],entry['Close']]);
let params={
 'names':['Closing Prices','Moving Average'] 
};
[closingData.transpose(),movingAverageData.transpose()].plot('multi-line','chart',params)```

AutoRegressive (AR) Model #

The autoregressive model predicts a value based on a number of lagged values (previous observations).

Steps: #

  1. Define a function to calculate the coefficients for the AR model.
  2. Use these coefficients to make predictions.

Here’s a basic implementation:

1. Calculate AR Coefficients #

For simplicity, we’ll use the Yule-Walker equations to estimate the AR coefficients.

function calculateARCoefficients(series, p) {
  const n = series.length;
  let autoCovariance = [];
  for (let k = 0; k <= p; k++) {
    let sum = 0;
    for (let t = k; t < n; t++) {
      sum += series[t] * series[t - k];
    }
    autoCovariance.push(sum / n);
  }

  let R = [];
  for (let i = 0; i < p; i++) {
    R[i] = [];
    for (let j = 0; j < p; j++) {
      R[i][j] = autoCovariance[Math.abs(i - j)];
    }
  }

  let r = autoCovariance.slice(1, p + 1);
  return solveMatrix(R, r);
}

function solveMatrix(R, r) {
  // Simple matrix solver for small matrices using Gaussian elimination
  const n = R.length;
  for (let i = 0; i < n; i++) {
    let maxEl = Math.abs(R[i][i]);
    let maxRow = i;
    for (let k = i + 1; k < n; k++) {
      if (Math.abs(R[k][i]) > maxEl) {
        maxEl = Math.abs(R[k][i]);
        maxRow = k;
      }
    }
    for (let k = i; k < n; k++) {
      [R[maxRow][k], R[i][k]] = [R[i][k], R[maxRow][k]];
    }
    [r[maxRow], r[i]] = [r[i], r[maxRow]];

    for (let k = i + 1; k < n; k++) {
      let c = -R[k][i] / R[i][i];
      for (let j = i; j < n; j++) {
        if (i === j) {
          R[k][j] = 0;
        } else {
          R[k][j] += c * R[i][j];
        }
      }
      r[k] += c * r[i];
    }
  }

  let coeffs = Array(n).fill(0);
  for (let i = n - 1; i >= 0; i--) {
    coeffs[i] = r[i] / R[i][i];
    for (let k = i - 1; k >= 0; k--) {
      r[k] -= R[k][i] * coeffs[i];
    }
  }
  return coeffs;
}

2. AR Model Prediction #

Use the calculated coefficients to make predictions.

function predictAR(series, p, numPredictions) {
  const coeffs = calculateARCoefficients(series, p);
  let predictions = series.slice();

  for (let i = 0; i < numPredictions; i++) {
    let nextValue = 0;
    for (let j = 0; j < p; j++) {
      nextValue += coeffs[j] * predictions[predictions.length - j - 1];
    }
    predictions.push(nextValue);
  }
  return predictions.slice(series.length);
}

// Example usage
const data = [112, 118, 132, 129, 121, 135, 148, 148, 136, 119, 104, 118, 115, 126, 141, 135, 125, 149, 170, 170, 158, 133, 114, 140, 145, 150, 178, 163, 172, 178];

const p = 2; // Number of lag observations (AR order)
const numPredictions = 5; // Number of future points to predict

const forecast = predictAR(data, p, numPredictions);
console.log('Forecast:', forecast);

Explanation: #

  1. calculateARCoefficients: This function calculates the AR coefficients using the Yule-Walker equations and a simple Gaussian elimination method for solving linear equations.
  2. predictAR: This function uses the calculated coefficients to predict future values based on the AR model.

This simple AR model can predict future values based on the linear combination of the past p values. While this implementation is basic, it illustrates the core concepts of autoregression. For more sophisticated models, especially for larger datasets and more complex time series, it’s advisable to use specialized libraries or statistical software.

Finally Calling the Autoregssion Function and Plotting the Result #

const closingPrices = timeSeriesData.map(entry => parseFloat(entry['Close']));
const p = 2; // Number of lag observations (AR order)
const numPredictions = 5; // Number of future points to predict

const forecast = predictAR(closingPrices, p, numPredictions);
forecast.plot()

Experiment With Time-series #

The Scribbler JavaScript notebook is an ideal tool for time series analysis due to its interactive and flexible environment. It allows users to write and execute JavaScript code directly in their web browser, making it highly accessible and easy to use. With the ability to import and utilize various JavaScript libraries, such as mljs, simple-statistics, and TensorFlow.js, it supports comprehensive data analysis and machine learning tasks. This interactivity is particularly beneficial for time series analysis, as users can visualize data trends, perform transformations, and apply models like ARIMA or autoregressive models in real-time. Additionally, the notebook’s capability to display graphical outputs and charts inline enhances the analytical process, providing immediate visual feedback and aiding in the intuitive understanding of complex time series data. By combining code execution, data visualization, and the power of modern JavaScript libraries, Scribbler JavaScript notebook offers a robust platform for conducting detailed and insightful time series analysis.

For experimenting in time-series use this notebook: Time-Series Analysis Notebook