Okay.
this is Discrete Optimization.
Mail Bag week 9, so this is the last time
we are taping.
Okay?
we are happy, as you can imagine, I mean
this went
really well, we are really excited by what
you guys did.
And so what we're going to do today is
basically summarize a bunch of things.
look at a lot of the data that we have on
the submission log on the survey.
And we're going to talk about assignments,
we're
going to talk about the various
optimization topics.
Which one you like?
Which one dislike?
I mean you like the all, right?
and then the student body hold you know,
what will be happen in the class.
we were lot of graphs, we're going to go
through them reasonably quickly.
this is the certificate that you know, a
curve showing you
the, the progression of the grace and the
certificate at the end.
I think I want to give you the context, so
I've
been teaching you an optimization class
for about 15 years.
Over these 15 years I have probably
graduated you know, 450 people.
And now we can look at what we have done
in the last nine weeks.
Okay?
So in the last not, nine weeks,
essentially, there has being
you know, about 2000 people who completed
you know, the knapsack assignment.
That's point A over there, right?
So, all of these people, okay?
So 790, you know, about 800 okay, which is
about 42% of this people okay 42% got a
certificate at the end okay.
So 800 compared to 450 you know, wow,
okay, nine weeks, okay.
So I should be resting for the next 15
years essentially.
Actually 30 years.
Okay.
And so certificate with distinction.
This is amazing as well, alright?
So we have you know, 378 so that's 48% of
this certificate.
So 48% of the certificate have
distinction.
So that for us really amazing.
You guys did an amazing job.
So this is curve you know, what we are
looking at every night when we are
depressed, okay.
this is the participation per assignment.
You have seen them a lot during the mail
bags nothing very new here.
You know, you can look at the VRP stuff,
VRP stuff
we would lost a couple of people, but very
few okay.
So essentially you know, this was very
stable once people actually succeeded
doing Knapsack.
And then after coloring we essentially
kept almost everybody.
this is a very interesting curve Okay.
So, so, you know, one of the things we did
here is dump the whole material at the
beginning for you.
You could do all the, you could watch, you
could watch all video, you
could watch assignments at anytime you
know,
and also do the assignment at any time.
And this is essentially trying to tell you
what
people have done with respect to the
assignment over time.
Okay?
So, there is a color for every one of the
assignments.
by now you probably know that I'm color
blind,
so I don't, I don't really know these
colors.
But I think knapsack is, if I remember
correctly, the yellow
you know, draft coloring is blue, TSP
should be something like orange.
Warehouse location, green.
And then the c are, the,
the, the VRP is black.
So if you think of a reason to like, like
this.
Okay?
So what you see is this nice shift.
Okay?
People after, after, after sometime, after
a couple,
after every week they would start a new
assignment.
And this is you know, to be expected.
That's what the you know, the assignments
were laid out for.
But what you are also seeing this is on
this side of this graph.
And which is really interesting.
Right?
Is that people kept working on the
assignments.
Okay?
They kept working
on coloring until the very end.
They kept working on warehouse else
location.
They kept working on the TSP until the
very end.
And this is amazing.
This is one of the things that we wanted
to do.
Right?
Once you know more, you can go back to an
assignment and improve it, and this
actually happened in the class.
And so, we, this is kind of validation for
us.
Dumping the whole material at the
beginning
of the class was a good decision.
this is more information about this, so
what you see here
is essentially what students did.
What tip, you know, the, the, the overall
class
did as far as, as the transition between
the assignment.
Okay?
So typically they would start at one.
Right?
So this is the knapsack.
Then they would move to coloring, they
would move to TSP.
They would move to warehouse location.
And then the VRP g/.
Okay?
So you expect this kind of linear, linear
model in a sense.
And the expectation here you know, this,
this, this,
the thickness of the line that you see
there
is one.
One student, right?
So, so, essentially that's what we
expected that the
students here would move, you know, for
these things.
What you can see is that of course, you
know, these lines here are thicker.
Okay?
Which basically means that people you
know,
you know, made these transitions more than
once.
Okay?
You also see that they went back a lot of
times to the other assignments.
Okay?
Not, not as many times as they went
forward obviously,
but people came back and, and, and
reconsidered the assignment
many times.
You see that particular, this is
particularly heavy here for
essentially the transition between VRP and
the traveling salesman problem.
Because I'm used to these two things, I'm
more connected.
So people probably learn more things on
the VRP.
They went back and proved the, the TSP and
so on.
Okay?
very interesting things here happening.
Okay?
And once again, this is basically telling
you that this open nature.
The fact that you reconsider the
assignments once you know more
about the technology.
You know, is actually very useful for
getting
better grades, for helping you understand
the material.
Okay?
So this is nice for us.
I mean, this means that we will continue
doing things like this.
assignment difficulty, you see the various
assignments here, and the difficulty.
No, no, no, no real surprise, maybe one
that I will mention here.
But essentially all the assignments are
hard or very hard.
You know, some of them are intermediate
for some people.
Knapsack is actually easy.
Okay?
Wish, once again was what we expected.
One of the big surprise for us is that
most of
the assignments seems to have a very
similar degree of difficulty.
And we were not expecting that.
Okay?
So for instance, we were expecting, I was
actually really expecting that VRP would
be a killer.
Okay?
And it was not.
Which basically means that people learn.
And there was a very nice discussion on
the forum.
Just expressing the fact that people learn
a lot of things when they
came to VRP.
And therefore the code of the VRP was
actually must smaller than the
[UNKNOWN]
of core, the, the, the, the code of, of
the TSP.
So very interesting for us.
Okay?
I was expecting you know, VRP to be really
difficult and in a sense, you
know, most of the students that learn the
material, and it was not that difficult.
Okay?
Which is great because, one, the VRP is,
the VRP assignment
was as close as we could to a real life
problem.
Okay?
So, very nice.
As, you know, assignment techniques for
people using practice.
Once again,
no big surprise for us.
You know, a lot of people use primarily
you know, heuristic local search.
This is to be expected.
We would like, you know, the constraint
programming
part and the MIP part to actually move up.
So we need to think about how to move that
for the next version of the
class, shows that people have you know,
broader
use of all the techniques in the
assignment.
Okay?
So that's for us, you know, to find, you
know, we need to find a way to do that.
but, but I think overall this is what
we expected.
we were expected for instance that there
would
be more people doing MIP on the warehouse
location.
What you see there, is a lot of people
using constraint parameter
for the graph, for graph coloring, that
was to be expected as well.
We expected that.
We expected the MIP column to be higher in
some of these assignments.
this area optim optimization topics that
you like and how much you like them.
Once again here, you know, pretty it's
very nice.
Okay?
Because it's mostly very consistent across
the various topic.
One other thing that, you know, is very
surprising to me is
that a lot of people love the look and
search lecture a lot.
so this is a surprise, because typically,
this is a tough topic to, to teach.
There are a lot of other, you know, the
nature of this topics is sometimes a
little bit anarchy.
Cannot be presented cohesively.
We spend a lot of time trying to design
the lecture on this.
And so, this is kind of nice to see that
actually
people really love those lectures.
So, good.
Excellent for us.
participation over time.
So, this is typically what you see on the
MOOC.
Right?
So, you start with a large number and that
number
[SOUND]
decreased.
And so that, that's, this is something
that you
see, but we have a lot more information
here.
Right?
So you see that a lot of the students who
were actually disappeared were students
that were really auditing the class.
You know, people were actually were very
active.
That means okay, I'm going to do the
assignments, right?
So you see that the decrease here was very
low.
About you know, we basically have a
retention from, you
know, the, after the knapsack to the end
of about 40%
which is quite nice for a very large scale
classes like this.
Okay?
once again, when you see these things that
are decreasing at the
end, you know, these are no, not
necessarily a real decrease here.
Alright?
Because some people they finished early,
and therefore, you know, it's
all obviously normal that fewer people are
working at the very end.
Okay?
So some people had finished really, really
early in the class.
You obviously don't see them working at
the very end.
So, once again, this is something that we
are you know, very happy.
You guys did an amazing job sticking with
the
class and we know that this was hard work.
Right?
and, and we had warned you about this.
this is also very interesting.
See this.
This is the participation of the survey.
A lot of people took the first survey.
But, then, you know, there was a big drop
to the mid-class survey.
But, now look, you know, the final survey
is almost at the same height.
And, this is very nice.
That means that, you know, from the, you
know, from, you know, the, the people who
were really active, you know, kept being
active until the very end.
Very, very nice for us.
Okay?
Now this is your goal and this is amazing.
Right?
So you get the expectation and actually
you met them.
Right?
Half of you said all we want a
certificate.
Half of you said you know we want a
certificate for distinction,
and look you know this is what we had at
the end, essentially.
These numbers really match the
certificates that we are actually
we, we are actually awarded at the end of
the class.
So beautiful,
you guys, you know, you knew what you
wanted.
You got what you wanted.
Okay?
Almost, right?
There are some exceptions, but very few.
Right?
So this is nice, very nice as well.
You guys did a great job.
Class difficulty, we told you it was hard
and it was hard.
Right?
So we were very transparent.
A couple of people thinks it's too hard, a
couple of people think this is really too
easy.
Yeah, to be expected again.
Right?
But this is, the class is hard, and that's
what we wanted to do.
Okay?
We told you we didn't water
down the topic, the, the, the material,
and you know, we expected essentially
this.
This is the workload, and I want to talk a
little bit about this, this is amazing as
well.
Okay?
So essentially you know, if we add,
continue this you
know, essentially this would, this would
have been a nice Gaussian.
Right?
So but we stop after 20 hours, we just
don't
want to know that somebody actually worked
50 hours a week.
Okay?
But, anyway.
So, this is a nice Gaussian
class Gaussian curve for the, the workload
of the class.
and there are two more things that, that
I want to say about this, one, three more
things.
So, one is that most MOOC are in this, in
this space.
That, you know, people spend less than 5
hours on the MOOC.
So, here the average, it should be around,
you know, 10 to 15 hours,
which is typically a class in a high
quality you know, university you know?
And, and so this is what you would spend
in those, in those settings.
So once again, we didn't water the
material
and you guys essentially spent the same
amount of
time that you know, people at top, top
level institution taking a class like this
would do.
So this is nice.
Obviously, what we need to do.
Okay?
And this is work for us, is try to
decrease a little bit the people who
are at the top you know, at the, at the,
at the right side of this curve.
Probably what we need to do is you know,
give some of the people
who are on that side more support for the
tools that they can use and
things like this.
Okay?
And this is one of the primary goal on the
next version of the class.
Now the other things that I wanted to say
about this curve is that any of you who
actually
spent less than an hour a week should give
me a phone call, we have a job for you.
Okay?
amount of learning.
Okay?
So this is once again, you know, the fact
that we are almost all on the
right here is great, so you know, a lot of
people, you know learn a lot.
Some people learn a huge amount
of material, this is nice.
Okay?
So good feedback for us once again.
overall experience Once again, this is
nice, especially given
the diversity of background that we have
in this class.
The fact that you know, this is really on
this side of this there will always be all
those people who, you know, don't like the
material
or don't like this teaching style and so
on.
But what you see here is most people got
you know, got a good experience in this
class.
And this is the typical ratio in a sense.
Except that
there is a bo, a little bit more diversity
here.
This is a typical ratio that we got for
the class you know, the campus, the campus
class.
This is almost the same kind of graph.
So this is also very nice.
So we have really replicated you know, the
qualities and the assessments, that the
evaluation that we had on the, for the
campus class into the MOOC class.
So which makes us very happy, because we
work like incredibly hard for this.
Right?
Oh, yeah, yeah, so this
is the language experience the programming
language experience
that people have had before coming to the
class.
And so this is an interesting curve.
Right?
So lot of people had experience in Python,
C++, Java,
these are typically the top three that,
that you expect.
but there is also a lot of experience on
other things like MATLAB, JavaScript, C #.
Okay?
And, and, and so the, there is a long
curve here.
Okay?
the other thing that you can see is
what is the, the languages that people
have used.
And this, this is for this person
who's name is unpronounceable, Mai, Maciej
or whatever.
So
[LAUGH]
people shouldn't have complicated name
like that, that should be forbidden.
Right?
Simple name.
Right?
But to, at least to pronounce.
Okay?
and so this is a, this is, this person
actually
asked us to give this, this resource at
the end.
So what other programming language that
people have used.
And so this is kind of interesting for us.
Okay?
So a lot of people here and you know they
love Python.
Okay?
They use Python.
A lot of people use C++ or Java.
Okay?
some C# over there.
Okay?
And there is this language for instance
called other, very popular too.
Right?
And so one of the things you know, that I
want, this is a free advertisement for
running language here.
Guys you should Objective C this is an
amazing language.
The syntax is a little bit funny, but this
is an amazing language.
Check it out.
Okay?
Just spend two hours checking that
language.
It's amazing.
Okay?
Okay.
use of general purpose solver.
You know, this is, this was actually an
interesting information for us.
Okay?
So it's more or less balanced.
Okay?
So more, you know, more people are not
using
it, which is essentially the intent of the
class originally.
And some of the people here are using it
as well.
So both, both are fine.
Okay?
I think in the future they will go, since
the class is targeting
a very broad audience we expect these
thing to be nicely balanced as well.
And we give most support to most people
actually using this tool so
that they can use them more effectively,
they learn them on their own.
But this is interesting.
This is interesting to see that this is
almost balanced.
Okay.
So this is a class that this is, this is,
this is
an interesting slide of what, whether you
would recommend this, this class.
And once again you know, we have a big,
you know, big, big, big
a large number of, of people, you know,
are very likely to recommend the class.
Now, you know, this is, this, one thing
that, you know, you
could do, you could, you know, all the
people in this category here.
Why don't you go to the sites, you know,
reviewing the class and, and just write a
review.
It doesn't have to be long.
You can just say this class is fantastic.
Hard work, but fantastic.
Right?
Just do that.
Okay?
That would be terrific for us.
Thank you.
and we hope to see you at the next
session.
Okay?
Hopefully, we can get some other people
who ordered it to actually do the
assignments.
And some of the people who almost complete
it, we hope to see you in March 2014.
That's where that's the next session of
this class.
And so the last you know, that you see
that most people were satisfied
with the content and what we have to make
it even better next time.
Okay.
Thank you very much guys.
You know, it was an incredible experience
for the team here, and
I hope you know, you enjoyed it, you
enjoyed it as much as
we did.
Thank you guys, bye.
Oh, I forgot my hat.
Okay?
I should say, ha, we have to re-tape
everything.
No we won't do it, but you know, thank you
guys.