1
00:00:03,780 --> 00:00:09,193
Okay. So let's continue our exploration 
of multiprocessor systems or small 

2
00:00:09,193 --> 00:00:13,949
multiprocessor systems. 
And we left off talking about concurrency 

3
00:00:13,949 --> 00:00:19,886
and one of the things we talked about the 
end of class or, or just a recap actually 

4
00:00:19,886 --> 00:00:26,021
before we go there that let's, let's talk 
about Sequential Consistency again as a 

5
00:00:26,021 --> 00:00:29,527
model. 
So Sequential Consistency as a model 

6
00:00:29,527 --> 00:00:34,284
basically says that you have lots of 
different threads executing. 

7
00:00:34,284 --> 00:00:40,211
And the ordering of the memory references 
from the different threads to be inner 

8
00:00:40,211 --> 00:00:44,016
leave but everyone has to agree on what 
ordering is. 

9
00:00:44,016 --> 00:00:49,846
And all the threads have to agree and see 
the same ordering and the ordering of the 

10
00:00:49,846 --> 00:00:54,317
loads in the stores in the respective 
memory references have to be some valid 

11
00:00:54,317 --> 00:00:57,985
ordering from the thread. 
So you can't reorder within the thread. 

12
00:00:57,985 --> 00:01:02,627
So if you have two threads and you have 
let's say try four fingers here and each 

13
00:01:02,627 --> 00:01:06,983
of your hand are for familiar references 
you can have some inter leaving. 

14
00:01:06,983 --> 00:01:10,996
You can have some other inter leaving. 
You can have that inter leaving. 

15
00:01:10,996 --> 00:01:15,409
You can have all these possible 
combinations leaving so what's not valid 

16
00:01:15,409 --> 00:01:19,014
is to have my two fingers sort of change 
order there and that is that's 

17
00:01:19,014 --> 00:01:22,769
sequentially consistent. Having said 
that, we talked about building processors 

18
00:01:22,769 --> 00:01:26,671
up to this point which are not do not 
maintain that order, so we talked about 

19
00:01:26,671 --> 00:01:30,572
out of order processors and out of order 
memory systems, which by definition are 

20
00:01:30,572 --> 00:01:33,060
not doing this sequentially consistent 
notion here. 

21
00:01:33,060 --> 00:01:36,824
And if you want performance, as we talked 
about, sometimes you want to move loads 

22
00:01:36,824 --> 00:01:40,494
around, you want to push the load up so 
you can get the, the operation out to the 

23
00:01:40,494 --> 00:01:43,412
memory system early. 
You try to push the store down because, 

24
00:01:43,412 --> 00:01:47,035
you haven't computed the result yet, and 
you want to be able to, sort of, compute 

25
00:01:47,035 --> 00:01:49,764
the result before you go push it out to 
the memory system. 

26
00:01:49,764 --> 00:01:53,529
So these things fight against each other. 
Having a strict memory model, something 

27
00:01:53,529 --> 00:01:56,305
like sequential consistency, and having 
good performance. 

28
00:01:56,305 --> 00:01:59,030
And in fact, 
you're probably not going to find any 

29
00:01:59,030 --> 00:02:02,979
processor which actually implements, true 
sequential consistency. 

30
00:02:02,979 --> 00:02:04,742
or. 
I'm trying to think. 

31
00:02:04,742 --> 00:02:09,683
Maybe the, one of the, I, I, I, I don't 
want to say no because I'm, I, I have a 

32
00:02:09,683 --> 00:02:15,182
hunch that the original, one of the 
original shared memory processes, the RP3 

33
00:02:15,182 --> 00:02:19,289
by IBM I believe might have had a very 
strict memory model. 

34
00:02:19,289 --> 00:02:22,630
But, besides that I don't think it had 
caches. 

35
00:02:22,630 --> 00:02:26,180
most, most things do not come even close 
to this. 

36
00:02:27,900 --> 00:02:32,096
But we talk about sequencially 
consistent, or sequencial consistency as 

37
00:02:32,096 --> 00:02:35,643
a good model, because a programmer likes 
to think about this. 

38
00:02:35,643 --> 00:02:40,490
They like to think that when they write a 
piece of code what they execute happens 

39
00:02:40,490 --> 00:02:44,686
in order, and it happens in order 
relative to every other piece of code 

40
00:02:44,686 --> 00:02:50,272
happening on, on other threads. 
So, just to recap, what this ends up 

41
00:02:50,272 --> 00:02:55,316
happening, what this ends up doing is, 
sequencial consistency, we, we had talked 

42
00:02:55,316 --> 00:03:00,294
about having sort of ordering between 
loads in stores on the same processor 

43
00:03:00,294 --> 00:03:05,600
before, and you might, you can break that 
maybe if they're two different addresses. 

44
00:03:05,600 --> 00:03:10,022
But Sequential Consistency adds 
additional arcs in our dependency graph 

45
00:03:10,022 --> 00:03:14,881
where every memory instruction in one 
thread is dependence on all the previous 

46
00:03:14,881 --> 00:03:18,307
memory instructions. 
That's one way to reason about not 

47
00:03:18,307 --> 00:03:23,041
reordering anything so if you introduce 
all those arcs you are not going to 

48
00:03:23,041 --> 00:03:25,657
accidentally break Sequential 
Consistency. 

49
00:03:25,657 --> 00:03:30,391
But sequential consistency says nothing 
about dependence from one thread to 

50
00:03:30,391 --> 00:03:34,690
another thread so there is no arc 
crossing between these two threads. 

51
00:03:34,690 --> 00:03:37,680
So hence you have all valid inter leaving 
there.