The Fei's Tech Notes: November 2013

Tuesday, November 26, 2013

ZFS Tuning

Check Arc Memory Usage
# echo ::memstat|mdb -k
Page Summary                Pages                MB %Tot
------------     ---------------- ---------------- ----
Kernel                     525663              4106    6%
ZFS File Data             5375945             41999   64%              <--- ZFS ARC Cache
Anon                      1641742             12826   20%
Exec and libs               35704               278    0%
Page cache                 531177              4149    6%
Free (cachelist)           121617               950    1%
Free (freelist)            156760              1224    2%
Total                     8388608             65536

to limit ZFS ARC Cache size, add/modify below line in /etc/system (it will take effect after next reboot)

set zfs:zfs_arc_max=6442450944

References:
http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide
http://docs.oracle.com/cd/E26502_01/html/E29022/chapterzfs-1.html#scrolltoc

Thursday, November 21, 2013

Blueimp Jquery Upload File Plugin

Change upload folder

<?php

* jQuery File Upload Plugin PHP Example 5.14

* https://github.com/blueimp/jQuery-File-Upload

* https://blueimp.net

* Licensed under the MIT license:

* http://www.opensource.org/licenses/MIT

define('DIR_DOWNLOAD', $_SERVER['DOCUMENT_ROOT'].'/');

define ('HTTP_SERVER' , 'http://'.$_SERVER['HTTP_HOST']);

error_reporting(E_ALL | E_STRICT);

require('UploadHandler.php');

$session_folder='aniu';

$upload_handler = new UploadHandler(array(

'upload_dir' => DIR_DOWNLOAD . 'userprofiles/',

'upload_url' => HTTP_SERVER . '/userprofiles/',

'image_versions' => array(

'thumbnail' => array(

'upload_dir' => DIR_DOWNLOAD . 'userprofiles/',
'upload_url' => HTTP_SERVER . '/userprofiles/',

));

UploadHandler

use index.php or specify a name

$(function () {
    'use strict';
    // Change this to the location of your server-side upload handler:
    var url = 'server/php/',
        uploadButton = $('<button/>')
            .addClass('btn btn-primary')
            .prop('disabled', true)
            .text('Processing...')
            .on('click', function () {

Change progress bar color

        .progress {
                height: 20px;
                margin-bottom: 20px;
                overflow: hidden;
                background-color: #f5f5f5;
                display: block;
                border-radius: 4px;
                box-shadow: inset 0 1px 2px rgba(0,0,0,0.1) }

        .progress-bar {
                float: left;
                overflow: hidden;
                display: block;
                width: 0;
                height: 100%;
                font-size: 12px;
                color: #008;
                text-align: center;
                background-color: #00fa0f;
                box-shadow: inset 0 -1px 0 rgba(0,0,0,0.15);
                transition: width .6s ease }

Change progress bar behaviour

        $('#progress .progress-bar').css(
            'visibility',
            'hidden'
        );

Remove "no file selected."

<!DOCTYPE html>
<html>
<head>
<style>

.inputWrapper {
    height: 100%;
    width: 108px;
    overflow: hidden;
    position: relative;
    cursor: pointer;
    /*Using a background color, but you can use a background image to represent a button*/
    background-color: #DDF;
}
.fileInput {
    cursor: pointer;
    height: 100%;
    position:absolute;
    top: 0;
    right: 0;
    z-index: 99;
    /*This makes the button huge. If you want a bigger button, increase the font size*/
    font-size:50px;
    /*Opacity settings for all browsers*/
    opacity: 0;
    -moz-opacity: 0;
    filter:progid:DXImageTransform.Microsoft.Alpha(opacity=0)
}
</style>
</head>

<body>
<div class="inputWrapper fileinput-button">
    <button id="image_alt">Select Picture</button>
    <input id="fileupload" class="fileInput" type="file" name="files[]" multiple>
</div>
</body>
</html>

Change File Name

<?php
/*
* jQuery File Upload Plugin PHP Example 5.14
* https://github.com/blueimp/jQuery-File-Upload
*
* Copyright 2010, Sebastian Tschan
* https://blueimp.net
*
* Licensed under the MIT license:
* http://www.opensource.org/licenses/MIT
*/

define('DIR_DOWNLOAD', $_SERVER['DOCUMENT_ROOT'].'/');
define ('HTTP_SERVER' , 'http://'.$_SERVER['HTTP_HOST']);

error_reporting(E_ALL | E_STRICT);
require('UploadHandler.php');
//$upload_handler = new UploadHandler();

$session_folder='aniu';

class CustomUploadHandler extends UploadHandler
{
    protected function trim_file_name($name, $type) {
        $name = parent::trim_file_name($name, $type);
        $name="thisistestname.jpg";
        return $name;
    }
}

$upload_handler = new CustomUploadHandler(array(
'upload_dir' => DIR_DOWNLOAD . '../test/files/',
'upload_url' => HTTP_SERVER . '/test/files/',
'image_versions' => array(
'thumbnail' => array(
'upload_dir' => DIR_DOWNLOAD .'/test/files/thumbnails/',
'upload_url' => HTTP_SERVER . '/test/files/',
),
),
));
?>

Monday, November 18, 2013

Python: Install Easy_install on Linux

wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | python

If failed to install, but you have got the binary file, you can try below command lines to finish the process

# python setup.py build

# python setup.py install

Wednesday, November 6, 2013

Install GIT Client on SuSE 11.2

Add repositories:
For Git,
# zypper addrepo http://download.opensuse.org/repositories/devel:/tools:/scm/SLE_11_SP2/devel:tools:scm.repo
For perl
# zypper addrepo http://download.opensuse.org/repositories/devel:/languages:/perl/SLE_11/devel:languages:perl.repo

if cannot find the repository, try below one.

# zypper addrepo http://download.opensuse.org/repositories/devel:/languages:/perl/SLE_11_SP3/devel:languages:perl.repo

Install packages
# zypper install perl-Error

# zypper install git-core

Tuesday, November 5, 2013

MongoDB: Sharding Database

#start shardsrv
mongod --shardsvr --dbpath ./data/db --port 27018 --smallfiles --logappend
#start configsrv
with --configsvr specified the default port for listening becomes 27019 and the default data directory /data/configdb. Wherever your data directory is, it is suggested that you verify that the directory is empty before you begin.
mongod --configsvr --dbpath ./data/configdb --smallfiles --logappend
#start mongos
mongos --configdb localhost:27019

#connect mongos
mongo
sh.addShard("a/localhost:27018")

to shard a collection, you must have an index on the shard key, so you will need to create the index first

db.trades.ensureIndex( { ticker:1, time:1 } )
sh.enableSharding("week6")
sh.shardCollection("week6.trades",{ticker:1,time:1},false)
use config
db.chunks.find()
db.chunks.find({}, {min:1,max:1,shard:1,_id:0,ns:1})

#start new monogd
mkdir ./data/db2
mongod --shardsvr --dbpath ./data/db2 --port 27020 --smallfiles --logappend
#add Shard in mongos prompt
sh.addShard("a/localhost:27020")
Now wait for the balancer to move data among the two shards more evenly. Periodically run: use config
db.chunks.find( { ns:"week6.trades" }, {min:1,max:1,shard:1,_id:0} ).sort({min:1})
db.chunks.aggregate( [
{ $match : { ns : "week6.trades" } } ,
{ $group : { _id : "$shard", n : { $sum : 1 } } }
] )

Build MongoDB Replica Set

Prepare DB Directories

mkdir -p /data/rs1 /data/rs2 /data/rs3

Configure Replica Set

Now start three mongo instances as follows. Note that are three commands. The browser is probably wrapping them visually.

./mongod --replSet ing --logpath "1.log" --dbpath /data/rs1 --port 27017 --smallfiles --fork
./mongod --replSet ing --logpath "2.log" --dbpath /data/rs2 --port 27018 --smallfiles --fork
./mongod --replSet ing --logpath "3.log" --dbpath /data/rs3 --port 27019 --smallfiles --fork

Now connect to a mongo shell and make sure it comes up

./mongo --port 27017

Now you will create the replica set. Type the following commands into the mongo shell:

config = { _id: "ing", members:[
          { _id : 0, host : "localhost:27017"},
          { _id : 1, host : "localhost:27018"},
          { _id : 2, host : "localhost:27019"} ]
};
rs.initiate(config);

At this point, the replica set should be coming up. You can type

rs.status()

to see the state of replication.

config = { _id: "ing", members:[
          { _id : 0, host : "localhost:27001"}
]
};
rs.initiate(config);

Other Replicaset Operation

rs.add("localhost:27002")
rs.add("localhost:27003")

rs.remove("localhost:27001")

Friday, November 1, 2013

Subversion Configuratioin Sample

Add below entries into /etc/apache2/conf.d/subversion.conf

<IfModule mod_dav_svn.c>
<Location /test>
        DAV svn
        SVNPath /srv/svn/test
        AuthType Basic
        AuthName "Test Repository"

        AuthzLDAPAuthoritative Off
        AuthBasicProvider ldap

        AuthLDAPURL "ldap://ldap.goweekend.ca/dc=goweekend,dc=ca?uid?sub?(ldappermission=svntest)"
        REQUIRE valid-user

        <LimitExcept GET PROPFIND OPTIONS REPORT>
          Require valid-user
        </LimitExcept>
</Location>

<Location /training>
        DAV svn
        SVNPath /srv/svn/training/

        Require valid-user
        <LimitExcept GET PROPFIND OPTIONS REPORT>
        AuthType Basic
        AuthName "SVN Training"
        AuthUserFile /srv/svn/training/conf/svnpasswd
        AuthzSVNAccessFile /srv/svn/training/conf/authz
        </LimitExcept>
</Location>

</IfModule>