[MUSIC]
Okay, so that was just one example of how we can think about looking at features of
a single input, but there are lots of other examples.
And let's just go through one application in particular where this is very useful,
and that's in detrending time series.
So here what I'm showing, is I'm showing house sales, which are these gray dots,
and there is a whole bunch of them, this is a real data set.
So lots and lots of house sales over time.
So instead of plotting house sales versus square feet,
here we're looking at the trends in house value over time.
And this plot in particular is for the Seattle metropolitan region and
this is some data that you guys have been playing around with
throughout this specialization.
And what this black curve shows here is the average house value versus time.
So just to be very specific our observation or
our output YI is the sales price of the ith, and
our input is going to be the time of that house sale.
So we're going to denote that by T sub i for the ith house.
And the time is recorded monthly because house
sales are recorded monthly at least in the US.
And one thing that we see is that on average,
the value of houses tends to increase with time.
So that's one effect that we probably want to capture.
But then there's another more subtle effect that might be hard to see from this
plot, but it's the fact that most houses are listed for
sale in the summer, that's the common housing season in the US.
And the good houses, they go really quickly.
In contrast, in for example November, December,
especially here in rainy Seattle, very few houses are listed for sale.
Very few people are going out shopping for houses in the rain.
So what ends up happening is any transactions that you see during these
months are really from leftover inventory
that was sitting around from the summer and just didn't sell in the summer because
they weren't the best houses during that really competitive period.
But if people are desperate and need to buy a house in November,
December they're left with whatever inventory is there.
Or there's some other special circumstance for why that house sale is going on.
And so the result of this is the fact that you tend to see
higher prices in the summer and lower prices in some of the off months.
And so what that means is what there's, what's called seasonality, okay.
Seasonality is the effect where over some period of time.
Which in this case is over the course of months.
We see an effect where there's repeated pattern of prices increasing,
decreasing, increasing, decreasing.
So this is something that'd we like to model.
And the way in which we're gonna model this is as follows.
We're gonna assume that our ith house sales, the price of that house sale
is comprised of the following different components.
There's one component which models just this increasing trend over time and for
the sake of this slide we're just assuming a very simple linear trend so this part
is just our simple linear regression model where our input Is this time index, ti.
Then though we're going to add some more features to this model and
the feature that we're going to add is this sinusoidal component which is
capturing the seasonality this fluctuation of increase prices in the summer,
decrease in the off seasons.
And what we want is we want this sinusoid to reset every year
because we see this pattern repeated again and again every year in our data.
So we're going to chose the period of the sinusoid, and
if you don't remember you're trigonometry that's okay.
Just look at this picture of the sinusoid.
That will be sufficient for what I'm talking about.
But idea is that it resets every 12 months and so
that's captured by this two pi t over 12 term.
And then the issues though is that in general in this case I've talked about
the fact that prices tend to increase in the summer months and
decrease In off season months, but
in general you don't really know where that seasonality trend occurs.
So there's some phase, some unknown phase to this process.
We can think of that just as a shift, and
that's represented by that phi parameter there.
I've marked in blue because that's the color of our parameters.
And just to animate this here, what I'm saying is we don't know where
this sinusoidal kind of trend is appearing in our data,
whether the peaks occur in June or January or something like this.
Okay, but now we have an issue because one of our parameters,
this phi term appears within this function of our input, ti.
So, we haven't yet talked about this, but what this means is that we're,
in this form, it's no longer just a simple linear regression where the parameters or
weights of our model are just multiplying the inputs or the functions of the input.
So it looks more complicated.
But there's a nice trick we can do here, and
that again is going back to some trigonometry.
Specifically the following trigonometric identity.
Which says that if you take sine of A-B, then that's equivalent
to sine(a)cosine(b)- cosine(a)sine(b).
Hopefully I got that ordering correct.
And if you apply that identity specifically to the case we have here what
you see is the following where
this phi parameter now what we have are two multiplicative terms.
We have a cosine phi and a sine phi or really a negative sine phi
multiplying the functions of our input Ti, which are shown in orange.
And so we can think of cosine of phi and
minus sine phi as just some W parameters in our model.
And that's summarized right here.
So an equivalent way to represent the model that we had on this slide here is as
follows, where we have again this linear term and then this sinusoidal component
we're breaking up into a sine and cosine term with these linear multipliers,
W2 and W3, to account for this unknown shift or phase to this function.
So, to make this very concrete, again we're in a featurized
situation where the first feature of our model is just that constant feature.
The second feature is just a linear feature, t itself, our input.
But the third feature and fourth feature are these sine and
cosine functions of our input, t.
Okay, so let's apply this model to our housing data.
And here in this plot we've done just that, we're fitting a polynomial
trend to capture this increase in prices over time as well as the sinusoidal
seasonal component to capture these fluctuations of prices with the season.
And so that's what's shown in this dark blue line here and in particular
in this case, instead of a simple linear trend, we fit a 5th order polynomial.
That's why we get a little bit more interesting of a shape over time.
And to see the effect of this sine cosine basis,
these features, let's zoom into this plot.
So now I've just zoomed in on a little chunk of this data and
you really see that sine and cosine having an effect with prices going up and
down over the course of seasons across these different years.
[MUSIC]