Okay. this is Discrete Optimization.
Mail Bag week 9, so this is the last time we are taping.
Okay?
we are happy, as you can imagine, I mean this went
really well, we are really excited by what you guys did.
And so what we're going to do today is basically summarize a bunch of things.
look at a lot of the data that we have on the submission log on the survey.
And we're going to talk about assignments, we're
going to talk about the various optimization topics.
Which one you like? Which one dislike?
I mean you like the all, right?
and then the student body hold you know, what will be happen in the class.
we were lot of graphs, we're going to go through them reasonably quickly.
this is the certificate that you know, a curve showing you
the, the progression of the grace and the certificate at the end.
I think I want to give you the context, so I've
been teaching you an optimization class for about 15 years.
Over these 15 years I have probably graduated you know, 450 people.
And now we can look at what we have done in the last nine weeks.
Okay?
So in the last not, nine weeks, essentially, there has being
you know, about 2000 people who completed you know, the knapsack assignment.
That's point A over there, right? So, all of these people, okay?
So 790, you know, about 800 okay, which is
about 42% of this people okay 42% got a certificate at the end okay.
So 800 compared to 450 you know, wow, okay, nine weeks, okay.
So I should be resting for the next 15 years essentially.
Actually 30 years.
Okay. And so certificate with distinction.
This is amazing as well, alright?
So we have you know, 378 so that's 48% of this certificate.
So 48% of the certificate have distinction.
So that for us really amazing.
You guys did an amazing job.
So this is curve you know, what we are
looking at every night when we are depressed, okay.
this is the participation per assignment.
You have seen them a lot during the mail bags nothing very new here.
You know, you can look at the VRP stuff, VRP stuff
we would lost a couple of people, but very few okay.
So essentially you know, this was very
stable once people actually succeeded doing Knapsack.
And then after coloring we essentially kept almost everybody.
this is a very interesting curve Okay.
So, so, you know, one of the things we did
here is dump the whole material at the beginning for you.
You could do all the, you could watch, you could watch all video, you
could watch assignments at anytime you know,
and also do the assignment at any time.
And this is essentially trying to tell you what
people have done with respect to the assignment over time.
Okay?
So, there is a color for every one of the assignments.
by now you probably know that I'm color blind,
so I don't, I don't really know these colors.
But I think knapsack is, if I remember correctly, the yellow
you know, draft coloring is blue, TSP should be something like orange.
Warehouse location, green.
And then the c are, the,
the, the VRP is black.
So if you think of a reason to like, like this.
Okay? So what you see is this nice shift.
Okay?
People after, after, after sometime, after a couple,
after every week they would start a new assignment.
And this is you know, to be expected.
That's what the you know, the assignments were laid out for.
But what you are also seeing this is on this side of this graph.
And which is really interesting. Right?
Is that people kept working on the assignments.
Okay?
They kept working
on coloring until the very end.
They kept working on warehouse else location.
They kept working on the TSP until the very end.
And this is amazing.
This is one of the things that we wanted to do.
Right?
Once you know more, you can go back to an
assignment and improve it, and this actually happened in the class.
And so, we, this is kind of validation for us.
Dumping the whole material at the beginning
of the class was a good decision.
this is more information about this, so what you see here
is essentially what students did.
What tip, you know, the, the, the overall class
did as far as, as the transition between the assignment.
Okay?
So typically they would start at one. Right?
So this is the knapsack.
Then they would move to coloring, they would move to TSP.
They would move to warehouse location.
And then the VRP g/.
Okay?
So you expect this kind of linear, linear model in a sense.
And the expectation here you know, this, this, this,
the thickness of the line that you see there
is one. One student, right?
So, so, essentially that's what we expected that the
students here would move, you know, for these things.
What you can see is that of course, you know, these lines here are thicker.
Okay?
Which basically means that people you know,
you know, made these transitions more than once.
Okay?
You also see that they went back a lot of times to the other assignments.
Okay?
Not, not as many times as they went forward obviously,
but people came back and, and, and reconsidered the assignment
many times.
You see that particular, this is particularly heavy here for
essentially the transition between VRP and the traveling salesman problem.
Because I'm used to these two things, I'm more connected.
So people probably learn more things on the VRP.
They went back and proved the, the TSP and so on.
Okay? very interesting things here happening.
Okay?
And once again, this is basically telling you that this open nature.
The fact that you reconsider the assignments once you know more
about the technology.
You know, is actually very useful for getting
better grades, for helping you understand the material.
Okay?
So this is nice for us.
I mean, this means that we will continue doing things like this.
assignment difficulty, you see the various assignments here, and the difficulty.
No, no, no, no real surprise, maybe one that I will mention here.
But essentially all the assignments are hard or very hard.
You know, some of them are intermediate for some people.
Knapsack is actually easy.
Okay? Wish, once again was what we expected.
One of the big surprise for us is that most of
the assignments seems to have a very similar degree of difficulty.
And we were not expecting that. Okay?
So for instance, we were expecting, I was
actually really expecting that VRP would be a killer.
Okay? And it was not.
Which basically means that people learn.
And there was a very nice discussion on the forum.
Just expressing the fact that people learn a lot of things when they
came to VRP.
And therefore the code of the VRP was actually must smaller than the
[UNKNOWN]
of core, the, the, the, the code of, of the TSP.
So very interesting for us.
Okay?
I was expecting you know, VRP to be really difficult and in a sense, you
know, most of the students that learn the material, and it was not that difficult.
Okay?
Which is great because, one, the VRP is, the VRP assignment
was as close as we could to a real life problem.
Okay?
So, very nice.
As, you know, assignment techniques for people using practice.
Once again,
no big surprise for us.
You know, a lot of people use primarily you know, heuristic local search.
This is to be expected.
We would like, you know, the constraint programming
part and the MIP part to actually move up.
So we need to think about how to move that for the next version of the
class, shows that people have you know, broader
use of all the techniques in the assignment.
Okay?
So that's for us, you know, to find, you know, we need to find a way to do that.
but, but I think overall this is what
we expected.
we were expected for instance that there would
be more people doing MIP on the warehouse location.
What you see there, is a lot of people using constraint parameter
for the graph, for graph coloring, that was to be expected as well.
We expected that.
We expected the MIP column to be higher in some of these assignments.
this area optim optimization topics that you like and how much you like them.
Once again here, you know, pretty it's very nice.
Okay?
Because it's mostly very consistent across the various topic.
One other thing that, you know, is very surprising to me is
that a lot of people love the look and search lecture a lot.
so this is a surprise, because typically, this is a tough topic to, to teach.
There are a lot of other, you know, the
nature of this topics is sometimes a little bit anarchy.
Cannot be presented cohesively.
We spend a lot of time trying to design the lecture on this.
And so, this is kind of nice to see that actually
people really love those lectures. So, good.
Excellent for us. participation over time.
So, this is typically what you see on the MOOC.
Right?
So, you start with a large number and that number
[SOUND]
decreased.
And so that, that's, this is something that you
see, but we have a lot more information here.
Right?
So you see that a lot of the students who
were actually disappeared were students that were really auditing the class.
You know, people were actually were very active.
That means okay, I'm going to do the assignments, right?
So you see that the decrease here was very low.
About you know, we basically have a retention from, you
know, the, after the knapsack to the end of about 40%
which is quite nice for a very large scale classes like this.
Okay?
once again, when you see these things that are decreasing at the
end, you know, these are no, not necessarily a real decrease here.
Alright?
Because some people they finished early, and therefore, you know, it's
all obviously normal that fewer people are working at the very end.
Okay?
So some people had finished really, really early in the class.
You obviously don't see them working at the very end.
So, once again, this is something that we
are you know, very happy.
You guys did an amazing job sticking with the
class and we know that this was hard work.
Right?
and, and we had warned you about this. this is also very interesting.
See this.
This is the participation of the survey. A lot of people took the first survey.
But, then, you know, there was a big drop to the mid-class survey.
But, now look, you know, the final survey is almost at the same height.
And, this is very nice.
That means that, you know, from the, you know, from, you know, the, the people who
were really active, you know, kept being active until the very end.
Very, very nice for us.
Okay?
Now this is your goal and this is amazing. Right?
So you get the expectation and actually you met them.
Right?
Half of you said all we want a certificate.
Half of you said you know we want a certificate for distinction,
and look you know this is what we had at the end, essentially.
These numbers really match the certificates that we are actually
we, we are actually awarded at the end of the class.
So beautiful,
you guys, you know, you knew what you wanted.
You got what you wanted.
Okay?
Almost, right? There are some exceptions, but very few.
Right? So this is nice, very nice as well.
You guys did a great job.
Class difficulty, we told you it was hard and it was hard.
Right? So we were very transparent.
A couple of people thinks it's too hard, a
couple of people think this is really too easy.
Yeah, to be expected again.
Right?
But this is, the class is hard, and that's what we wanted to do.
Okay? We told you we didn't water
down the topic, the, the, the material,
and you know, we expected essentially this.
This is the workload, and I want to talk a
little bit about this, this is amazing as well.
Okay?
So essentially you know, if we add, continue this you
know, essentially this would, this would have been a nice Gaussian.
Right?
So but we stop after 20 hours, we just don't
want to know that somebody actually worked 50 hours a week.
Okay?
But, anyway. So, this is a nice Gaussian
class Gaussian curve for the, the workload of the class.
and there are two more things that, that
I want to say about this, one, three more things.
So, one is that most MOOC are in this, in this space.
That, you know, people spend less than 5 hours on the MOOC.
So, here the average, it should be around, you know, 10 to 15 hours,
which is typically a class in a high quality you know, university you know?
And, and so this is what you would spend in those, in those settings.
So once again, we didn't water the material
and you guys essentially spent the same amount of
time that you know, people at top, top
level institution taking a class like this would do.
So this is nice.
Obviously, what we need to do.
Okay?
And this is work for us, is try to decrease a little bit the people who
are at the top you know, at the, at the, at the right side of this curve.
Probably what we need to do is you know, give some of the people
who are on that side more support for the tools that they can use and
things like this. Okay?
And this is one of the primary goal on the next version of the class.
Now the other things that I wanted to say
about this curve is that any of you who actually
spent less than an hour a week should give me a phone call, we have a job for you.
Okay?
amount of learning. Okay?
So this is once again, you know, the fact that we are almost all on the
right here is great, so you know, a lot of people, you know learn a lot.
Some people learn a huge amount
of material, this is nice. Okay?
So good feedback for us once again.
overall experience Once again, this is nice, especially given
the diversity of background that we have in this class.
The fact that you know, this is really on this side of this there will always be all
those people who, you know, don't like the material
or don't like this teaching style and so on.
But what you see here is most people got
you know, got a good experience in this class.
And this is the typical ratio in a sense. Except that
there is a bo, a little bit more diversity here.
This is a typical ratio that we got for
the class you know, the campus, the campus class.
This is almost the same kind of graph. So this is also very nice.
So we have really replicated you know, the qualities and the assessments, that the
evaluation that we had on the, for the campus class into the MOOC class.
So which makes us very happy, because we work like incredibly hard for this.
Right? Oh, yeah, yeah, so this
is the language experience the programming language experience
that people have had before coming to the class.
And so this is an interesting curve.
Right?
So lot of people had experience in Python, C++, Java,
these are typically the top three that, that you expect.
but there is also a lot of experience on other things like MATLAB, JavaScript, C #.
Okay?
And, and, and so the, there is a long curve here.
Okay?
the other thing that you can see is
what is the, the languages that people have used.
And this, this is for this person
who's name is unpronounceable, Mai, Maciej or whatever.
So
[LAUGH]
people shouldn't have complicated name like that, that should be forbidden.
Right?
Simple name. Right?
But to, at least to pronounce. Okay?
and so this is a, this is, this person actually
asked us to give this, this resource at the end.
So what other programming language that people have used.
And so this is kind of interesting for us.
Okay?
So a lot of people here and you know they love Python.
Okay? They use Python.
A lot of people use C++ or Java.
Okay? some C# over there.
Okay?
And there is this language for instance called other, very popular too.
Right?
And so one of the things you know, that I
want, this is a free advertisement for running language here.
Guys you should Objective C this is an amazing language.
The syntax is a little bit funny, but this is an amazing language.
Check it out.
Okay?
Just spend two hours checking that language.
It's amazing. Okay?
Okay.
use of general purpose solver.
You know, this is, this was actually an interesting information for us.
Okay?
So it's more or less balanced. Okay?
So more, you know, more people are not using
it, which is essentially the intent of the class originally.
And some of the people here are using it as well.
So both, both are fine.
Okay?
I think in the future they will go, since the class is targeting
a very broad audience we expect these thing to be nicely balanced as well.
And we give most support to most people actually using this tool so
that they can use them more effectively, they learn them on their own.
But this is interesting.
This is interesting to see that this is almost balanced.
Okay.
So this is a class that this is, this is, this is
an interesting slide of what, whether you would recommend this, this class.
And once again you know, we have a big, you know, big, big, big
a large number of, of people, you know, are very likely to recommend the class.
Now, you know, this is, this, one thing that, you know, you
could do, you could, you know, all the people in this category here.
Why don't you go to the sites, you know,
reviewing the class and, and just write a review.
It doesn't have to be long. You can just say this class is fantastic.
Hard work, but fantastic. Right?
Just do that.
Okay? That would be terrific for us.
Thank you.
and we hope to see you at the next session.
Okay?
Hopefully, we can get some other people
who ordered it to actually do the assignments.
And some of the people who almost complete it, we hope to see you in March 2014.
That's where that's the next session of this class.
And so the last you know, that you see that most people were satisfied
with the content and what we have to make it even better next time.
Okay. Thank you very much guys.
You know, it was an incredible experience for the team here, and
I hope you know, you enjoyed it, you enjoyed it as much as
we did. Thank you guys, bye.
Oh, I forgot my hat.
Okay?
I should say, ha, we have to re-tape everything.
No we won't do it, but you know, thank you guys.