Physics of Information - NiPS) Lab

Physics of Information

Igor Neri - July 17, 2018

NiPS Summer School 2018Energy aware transprecision computing

Limits to computation

• Minimum size of computing device

• Maximum computational speed of a self-contained system

• Information storage in a finite volume

• Energy consumption limit to:

• computation

• memory preservation

Minimum size of computing device

Transistor count

…

Moore’s law

Transistor size

“Alcune riflessioni sulla legge di Moore”, Roberto Saracco, Future Center, TILAB

In terms of size [of transistors] you can see that we're approaching the size of atoms which is a fundamental barrier, but it'll be two or three generations before we get that far - but that's as far out as we've ever been able to see. We have another 10 to 20 years before we reach a

fundamental limit. By then they'll be able to make bigger chips and have transistor budgets in the billions. - G. Moore

Energy limits speed of computation


• What limits the laws of physics place on the speed of computation?

Ultimate physical limits to computation - Seth Lloyd


Heisenberg uncertainty principle

wrong interpretation: it takes time ∆t to measure energy to an accuracy ∆E

right interpretation: a quantum state with spread in energy ∆E takes time at least

to evolve to an orthogonal (and hence distinguishable) state

�E�t � ~/2


Maximum computational speed of a self-contained system

• Bremermann's Limit is the maximum computational speed of a self-contained system in the material universe. It is derived from Einstein's mass-energy equivalency and the Heisenberg uncertainty principle, and is c2/h ≈ 1.36 × 1050 bits per second per kilogram

• The Margolus–Levitin theorem gives a fundamental limit on quantum computation (strictly speaking on all forms on computation). The processing rate cannot be higher than 6 × 1033 operations per second per joule of energy

Maximum computational speed of a laptop

?

Maximum computational speed of a laptop

• If the mass is m then E = mc2

• m = 1 Kg, E = 1 (3 108)2 = approx = 1017 J

• ∆t = approx = 10-34 /1017 = 10-51 s


Comparison with existing computers

• Conventional laptops operate much more slowly than the ultimate laptop

• Two reasons for this inefficiency:

• most of the energy is locked up in the mass of the particles of which the computer is constructed

• a conventional computer employs many degrees of freedom for registering a single bit


Memory space limits

Memory space limits

• The amount of information that a physical system can store and process is related to the number of distinct physical states accessible to the system

• A collection of M two-state systems has 2M accessible states and can register M bits of information

• A system with N accessible states can register log2N bits of information


Memory space limits

• The number of accessible state, W, of a physical system is related to its thermodynamic entropy by the formula: S = kB logW

• The amount of information that can be registered by a physical system is I = S(E)/kB log 2

• S(E) is the thermodynamic entropy of a system with expectation value for the energy E


Information storage in a finite volume

• The Bekenstein bound limits the amount of information that can be contained within a given finite region of space which has a finite amount of energy:

I 2⇡cRm

~ ln 2 ⇡ 2.577⇥ 1043mR

S 2⇡kRE

~c


Information storage in a finite volume

• Human brain

• mass m=1.5 kg

• volume of 1260 cm3

• approximating volume to a sphere R = 6.7 cm

• I = 2.6 x1042 bits

• O = 2I states of the human brain must be less than

I 2⇡cRm

~ ln 2 ⇡ 2.577⇥ 1043mR

⇡ 107.8⇥1041

Comparison with existing computers

• The amount of information that can be stored by the ultimate laptop ≈ 1031 bits

• Conventional laptops can store ≈ 1012 bits

• This is because conventional laptops use many degrees of freedom to store a bit where the ultimate laptop uses just one

• There are considerable advantages to using many degrees of freedom to store information, stability and controllability being perhaps the most important


Minimum energy consumption for computation

Landauer R. IBM Journal Of Research And Development, Vol. 5, no. 3, 1961

Information is

physical

Maxwell’s demon

Maxwell’s demon

Cyclic process converts heat completely into work!

Violates second law of thermodynamics!

No process is possible whose sole result is the absorption of heat from a

reservoir and the conversion of this heat into work.

Maxwell’s demon


Information is

physical

What happens when computation is logically irreversible?

• Minimum amount of energy required greater than zero

• Let assume the operation of bit reset

• # of initial states: 2

• # of final states: 1

Landauer principle

• Initial condition: two possible states

• Final condition: one possible state

• Heat produced

S = kB logW

Q T�S

Si = kB log 2

Sf = kB log 1

Q T�S = �kBT log 2

�S = Sf � Si = �kB log 2

Landauer principle at room temperature

Q T�S = �kBT log 2 ⇠ 10�21J

Landauer principle experimental verification

The physics of information: from Maxwell’s demon to Landauer - Eric Lutz - University of Erlangen-Nürnberg


Even if you're not burning books, destroying information generates heat. - Sergio Cicliberto


The physics of information: from Maxwell’s demon to Landauer - Eric Lutz - University of Erlangen-Nürnberg

Reset on colloidal particles

Chiuchiú, D. "Time-dependent study of bit reset." EPL (Europhysics Letters) 109.3 (2015): 30002.

Time-dependent study

For a fixed τpr with Q(τpr) ≈ −T∆S(τpr), study −T∆S(t), Q(t), W (t), ∆E(t).

Chiuchiú, D. "Time-dependent study of bit reset." EPL (Europhysics Letters) 109.3 (2015): 30002.


Jun, Y., Gavrilov, M., & Bechhoefer, J. (2014). High-Precision Test of Landauer’s Principle in a Feedback Trap. Physical Review Letters, 113(19), 190601.


Feedback Trap



Erasure protocol



Work series for individual cycles


Beating the Landauer's limit by trading energy with uncertainty

Beating the Landauer's limit by trading energy with uncertainty - L. Gammaitoni - arXiv:1111.2937 [cond-mat.mtrl-sci]

http://arxiv.org/abs/1111.2937

Beating the Landauer's limit by trading energy with uncertainty

Landauer limit

Beating the Landauer's limit by trading energy with uncertainty - L. Gammaitoni - arXiv:1111.2937 [cond-mat.mtrl-sci]

http://arxiv.org/abs/1111.2937

Micro-electromechanical memory bit based on magnetic repulsion

Micro-electromechanical memory bit based on magnetic repulsion, López-Suárez, Miquel and Neri, Igor, Applied Physics Letters, 109, 133505 (2016)

Micro-electromechanical memory bit based on magnetic repulsion

Orders of magnitude above Landauer limit!Micro-electromechanical memory bit based on magnetic repulsion, López-Suárez, Miquel and Neri, Igor, Applied Physics Letters, 109, 133505 (2016)

Solution: increase the temperature

Neri, Igor, and Miquel López-Suárez. "Heat production and error probability relation in Landauer reset at effective temperature." Scientific Reports 6 (2016).

Teff = 5 × 107 K

Reset protocol


Q=W-ΔU

Landauer reset with error


Logically irreversible devices


We shall call a device logically irreversible if the output of a device does not uniquely define the inputs. We believe that devices exhibiting

logical irreversibility are essential to computing. Logical irreversibility, we

believe, in turn implies physical irreversibility, and the latter is

accompanied by dissipative effects.

Information is Physical

Rolf Landauer, 1961. Whenever we use a logically irreversible gate we dissipate energy into the environment.

Logically irreversible devices

Bennett C. IBM Journal of Research and Development, vol. 17, no. 6, pp. 525-532, 1973

Landauer has posed the question of whether logical irreversibility is an unavoidable feature of useful

computers, arguing that it is, and has demonstrated the physical and philosophical importance of this

question by showing that whenever a physical computer throws away information about its previous

state it must generate a corresponding amount of entropy. Therefore, a computer must dissipate at

least kBT ln2 of energy (about 3 X 10-21 Joule at room temperature) for each bit of information it erases or

otherwise throws away.

Solution = Reversibility

• Charles Bennett, 1973: There are no unavoidable energy consumption requirements per step in a computer.

• Energy dissipation of reversible circuit, under ideal physical circumstances, is zero.

Reversible computation

• Landauer/Bennett: all operations required in computation could be performed in a reversible manner, thus dissipating no heat.

• The first condition for any deterministic device to be reversible is that its input and output be uniquely retrievable from each other, then it is called logically reversible.

• The second condition: a device can actually run backwards, then it is called physically reversible, and the second law of thermodynamics guarantees that it dissipates no heat.

Billiard ball computing

• Model of a reversible mechanical computer based on Newtonian dynamics

• Proposed in 1982 by Edward Fredkin and Tommaso Toffoli

• It relies on the motion of spherical billiard balls in a friction-free environment made of buffers against which the balls bounce perfectly


• Using balls and mirrors, we can implement basic logic gates: AND, OR, NOT

• With a big enough billiard table, we could (in theory) implement a complete computer using a combination of these gates

• BUT…

• billiard balls don't work in practice


• Thermal losses

• friction can't be ignored

• Collisions aren't perfectly elastic

• Chaotic motion

• Balls are actually conglomerates of many atoms in various states of vibration

• Can't know their “initial state” perfectly

• Small variations in initial conditional conditions can cause exponentially large differences in final state

Reversible computing

• The reasoning on connection between physical and logical reversibility applies only to systems that encode input and outputs on the system itself.

• If the input and output are not part of the computing system (like in transistor based logic gates) there is no connection between physical and logical reversibility.

OR gate

Back to the real world….

Ω0 Ω1

OR gate


OR gate

www.randomwraith.com


The experimental setup


Ω0 Ω1


Landauer limit

XOR gate

Full adder

Minimum energy consumption for memory preservation

The refresh procedure

Symbol to storeis known with no error

E. g. 1

Repeat times

Wait Refresh:

1) Read the current symbol2) Write the read symbol

Symbol is 1with probability

is a functionof and of therepetition number

Di usions of the physicalquantity that encodesinformation are removed

Symbol is 1with probability

After Refresh

After repetitions, theprobability that there areundesired transitions to 0

To evaluate the energy cost of the refresh procedure we need:

• A physical description of the memory

• A characterisation of P0 as function of refresh time tR

• A physical description of the refresh procedure

• A characterisation of total error probability PE as function of refresh time tR after a fixed time

Physical description of the memory

Characterisation of P0 as function of refresh time

0 0 0

0 0 0

a) b) c)

d) e) f)

@

@tp(x, t) =

@

@x

✓@U

@xp(x, t)

◆+

kBT

�U

@2

@x2p(x, t),

P0(t) =

Z 0

�1p(x, t)dx

Physical description of the refresh procedure

Characterisation of PE as function of refresh time

102 104 10610-7

10-6

10-5

10-4

10-3

10-2

10-1

100

10-0.5

100

100.5

101

101.5

102

100.2

100.1

100

10-0.1

10-0.5

What is the fundamental cost for preserving a memory for a fixed time with a given probability

of error?

Study of the energy cost of refresh procedure

�(tR) =

r�2w + exp

⇣� tR

⌧w

⌘(�2

i � �2w) �i

▸ Considering the harmonic approximation inside each well the refresh operation changes: in

Minimum energy required to preserve a memory over a fixed time with a given error probability

Qm = �NT�S =t

tRkBT ln

0

@

r

(�2w+e

� tR⌧w (�2

i��2w)

�i

1

A


PE=1×10−6 PE=1×10−4 PE=1×10−2

0 0.2 0.4 0.6 0.8 1

×108

0

0.5

1

1.5

2b)

0 0.5 1 1.5 2 2.5

×108

0

0.5

1

1.5

2

10-6 10-4 10-2 10 0105

1010

1015

1020a)


0 0.2 0.4 0.6 0.8 1

×108

0

0.5

1

1.5

2b)

0 0.5 1 1.5 2 2.5

×108

0

0.5

1

1.5

2

10-6 10-4 10-2 10 0105

1010

1015

1020a)

PE=1×10−6 PE=1×10−4 PE=1×10−2

Limits to computation

• Minimum size of computing device

• Maximum computational speed of a self-contained system

• Information storage in a finite volume

• Energy consumption limit to:

• computation

• memory preservation

References• Lloyd, Seth. "Ultimate physical limits to computation." Nature 406.6799 (2000): 1047.

• Aharonov, Yakir, and David Bohm. "Time in the quantum theory and the uncertainty relation for time and energy." Physical Review 122.5 (1961): 1649.

• Landauer, Rolf. "Irreversibility and heat generation in the computing process." IBM journal of research and development 5.3 (1961): 183-191.

• Toyabe, Shoichi, et al. "Experimental demonstration of information-to-energy conversion and validation of the generalized Jarzynski equality." Nature physics 6.12 (2010): 988.

• Bérut, Antoine, et al. "Experimental verification of Landauer’s principle linking information and thermodynamics." Nature 483.7388 (2012): 187.

• Chiuchiú, D. "Time-dependent study of bit reset." EPL (Europhysics Letters) 109.3 (2015): 30002.

• Lopez-Suarez, Miquel, Igor Neri, and Luca Gammaitoni. "Sub-k B T micro-electromechanical irreversible logic gate." Nature communications 7 (2016): 12068.

• Neri, Igor, and Miquel López-Suárez. "Heat production and error probability relation in Landauer reset at effective temperature." Scientific reports 6 (2016): 34039.

• López-Suárez, M., et al. "Cost of remembering a bit of information." Physical Review A 97.5 (2018).

Thank you for your attention!

[email protected]

Physics of Information - NiPS) Lab

Documents

Transcript of Physics of Information - NiPS) Lab