YAPC::Europe: Tuesday
it's just my notes, modified for
obvious spelling errors fixes and URLs for the interesting bits. It may
contains errors. I'll post proper and scoped articles later.
Upate: Fixed broken links
Larry Wall's Keynote
scripting
languages
past, present and future
ruby
most direct competitor for perl
perl 6 mix between
pure scripting and programming language
lua,
applescript: niche player
failed: tcl (due to lack
of extensibility), *sh (clumsy addition of layers of features)
early
binding vs late binding
perl6: all method are virtual by
default
single dispatch, multiple dispatch
single:
python, perl5
multiple: perl6, dylan
eager
or lazy evaluation
haskell: very lazy evaluation
perl6:
scalar is eager by default, list is lazy
eager
typology, lazy typology:
types introduced in perl6 for the
multiple dispatch
fixes e.g: prototyping
removed
from perl6 punctuation that were not really necessary
introduce
a new one for scoping
mutable, immutable classes
java
classes are immutable -> fast
ruby classes are mutable
-> ruby slow
perl6 will have a mix
class
based OO, prototype based OO
perl6 will be classed based, but
meta data will allow prototype based OO
(see Moose in perl5)
perl6:
given ... when ... for regexp?
Selenium,
an introduction to web testing (Barbie)
benefits over
Mechanize: test javascript as it runs
Thoughworks
released
as open source on openQA
use javascript and iframes
in the browser
core runs the tests and interrogates the DOM
RC
server and core communicate via AJAX
Core, Remote
Control, IDE (firefox plugin)
Core: issues with Opera
RC:
java, requires JRE version 1.5.0 or higher
experimental
support for SSL
language hooks for
java, .Net (C#),
Perl, PHP, Python, Ruby
Mozilla same origin policy
IDE:
record/playback, edit and debug tests
include Selenium Core
cpan>
install Alien::SeleniumRC
(cant
upgrade due to how versions are dealt with)
cpan>
install Test::WWW::Selenium
$
selenium-rc
use WWW::Selenium::Util
qw(server_is_running)
Evolving
architecture - make development easy and your site faster (Leo Lapworth)
evolution
running
website
- servers improved
architecture too
- development tools improved
language
- templates
hard
coded html in the beginning
now templates
Template
- servers
from development on 1 server to 3-tier servers
svk with subversion
trac
-
tests
-
make your site faster
mod_perl (code caching)
Apache::SizeLimit
(safety net)
-> set it high
-> check it
- front/back end split
(sees Omnigraffle schema)
add caching
search result sets
individual items
lookups for info from database
lookup from external sources
put caching methods in one packages
separate cache for each backend servers?
-> share them (using memcached)
perlbal
-> load balancer/proxy
mod_gzip
cache headers (expiry)
/includes/js/
forever
ensures user has version which matches html
use include file to update all pages
-
handling images: MogileFS
-
centralize
-
test
-
cache
-
kiss (esp. perlbal)
XML::Compile::SOAP (Mark Overmeer)
XML
sucks (verbose, looks simple but its not)
XML schema
WSDL,
SOAP
avoid learning XML and Schema
pure
perl, compliant, complete, validating, xml message reading and writing
use
XML::Compile::Schema;
good:
automatic
name-spaces
type structures hidden (inheritance etc)
template
generator
limitation:
only
name-space based schemas
mixed content only via hooks
schemas
themselves not validated
you need a schema to use
the module
SOAP (PayLoad - all XML -, Transport - in
application)
Payload = Body + Header ( Envelope)
two
kinds of SOAP:
Document
- well defined body
requires longer schemas
XML-RPC
*
interface quick and dirty
- SOAP::Lite
special - discouraged in SOAP 1.2
WSDL
message
structure and transports details are grouped together.
SOAP
client/server implementation still under construction
use
BigInt instead of sloppy int -> slight reduction in performances.
Gluing
a bank together (UBS) (Paul Johnson)
move lots of money around to
avoid interests or to gain interests
Cash management
CPAN
primary
development is outsourced
needs to customize the
product
needs to be integrated
database
web
servers
communications
high availability
monitoring
logging
archiving
deployment
initially
role is automated testing
perl as a development
language is not allowed in UBS
but perl to glue
thing together is Ok, then development could be done
Oracle
100s
of GB
-
Web server
IHS:
IBM re branded version of Apache -
Communication
multiple
sources
multiple format
-
message
transfer: how amount goes from what bank to what other bank in what
currency
IBM MQSeries -
mail
SMS
- IRC
- file transfer
pack
and unpack
use
MQSeries
; # written and maintained by Morgan Stanley people
system
handles many millions of money currency
if system
breaks, huge amount of money is lost
monitoring
-> Nagios
logs
50 GB a day
require application restart to log rotate
so
he write wrappers with named pipes, correct formatting including
timestamps
use Log::Log4Perl
Deployment:
Sun
packages
package creation
mini
CPAN burnt on CD
Extra development
internal
part base on Catalyst
(DBIx::Class,
Template
Toolkit)
Automated testing
Test::*
use Test::WWW::Selenium
Trexy (Nigel Hamilton)
trexy.com
remember
search trails
my trails - all trails - blaze a trial
30
millions incoming links
Sys::Statistics::Linux::MemStats
The
Goo
perceptrons
sensors
Tech
Pub Crawl: first Tuesday of the month in London
flag-and-bell.com
FREE BEER
memcached (Leon Brocard)
network
effect
-> scaling?
temporary storage
area fro frequently accessed data can be stored for rapid access
trade
memory/disk speed
One Server:
MySQL
query cached - invalidated on write
Disk - Cache::FileCache
scales
really well
memory bound
mod_perl
only
one per child
shared memory
not as fast as
you might think
cache is separate on each
lower
hit ratio
higher miss ratio
memcached
giant
hash table distributed across machines
never
blocks
libevent
epoll/kqueue
slab
allocator
least request used
thread
per cpu (optionally)
version 1.2.x are much better
facebook:
3TB memcached
use Cache::Memcached
Pattern:
fetch
from cache
if there return
else calculate, place in cache, return
cache, not a
database
-> cant dump
-> no
persistence
-> no redundancy
-> no
access by id
-> ...
time to live
smart
caching
timestamps, version number in key
cache
forever
low CPU
Failover?
doest
do it for you
replace failed server with another with same ip
or
use consistent hashing
limits:
keys: max
250 chars
values: max 1MB
Testing
*
disable memcached
future:
consistent
hashing
binary protocol
more statistics
http://www.danga.com/memcached/
has
to push the keys to all memcached servers
memcached, perlbal, mogileFS,
Djabberd,Gearman
TheSchwartz
Net::Proxy (Philippe Bruhat)
connectors
a
connector handles the pairs of socket (one for each client)
Use?
*
escape the corporate proxy
CONNECT method
(abuse of
the the CONNECT, normally for SSL?)
-
avoid
Intrusion Detection Systems -
early stage of ssh
negotiation is not encrypted and can be detected by IDS by doing a
m/ssh/ -
use hooks to hide ssh signature using one Net::Proxy
before the firewall and another Net::Proxy the other side of the
firewall to decrypt ssh signature -
add SSL support
to an application that doest support it -
run two
servers on the same port
we want to run
sshd and https on the same port -
in ssh negotiation, server speaks first
-
in http/ssl: client speaks first
-
Net::Proxy uses that to make it possible
todo:
-
write a connector fully
compatible with GNU httptunnel -
enhance the httptunnel protocol to support multiple connections.
-
implement reverse connectors (as you cannot connect to
machines
behind firewalls at the moment) -
implement DNS tunnel connectors
-
implement UDP connectors
-
implement a connector that can be plugged to the STDIN/STDOUT of an
external process, like the ProxyCommand option of OpenSSH -
finish the starttls connector
-
implement SOCKS connectors