2015年9月6日 星期日

Hadoop



Commetns:
- distributed architecture for compunting & storage
- could run in 3 modes : standalone , pseudo districuted, fully distributed
- client-server architect :
Server : run Name Node + Job Tracker
Client : run Data Node + Task Tracker
- the execution liverage counts on : MapReduce which is executed in slave side
-  HDFS is the filesystem used in hardoop, it will be kept in 3 different place. 2 is near-end side and the other is in far-end side
- HBASE is like a database could use for saving data after Reduce


Reference book:
- Windoop應用實作指南-掌握Hadoop翱翔雲端 by 許清榮,林奇暻, 買大誠


Thinking:
- if our traget data is distributed everywhere, maybe it's a way to handle locally and migrate.
- the point is map & key, the application must be divided & indexed via a key.

2015年5月4日 星期一

JNI coding

src:
http://blogforopensource.blogspot.tw/2012/08/jni-sample-java-code.html



notes:
15'5/5 :
- javah is used for JNI header
- pay attention to the c compile. It may need extra compiler like VC++




contents:

Blogging after a long time..


JNI - Java Native Interface.


Java interface to non­Java code.
It is Java's link to the "outside world"

•  Native methods are compiled into a dynamic link
  library (.dll, .so, etc.)
•  OS loads and links the library into the process
  that is running the Java Virtual Machine
•  Part of the Java Developer Kit(JDK),
  serves as a glue between Java side and
  native side of an application
• Allows Java code that runs inside a Java Virtual
  Machine (JVM) to inter operate with applications and
  libraries written in other programming languages,
  such as C, C++, and assembly

JNI w.r.t Operating System :


JNI Sequence:


If you are really interested in JNI,

You can find a Sample Code from here.

http://code.google.com/p/onlineexaminationsysteminjava/downloads/detail?name=samplejnicode.zip&can=2&q=

Reference:

http://patriot.net/~tvalesky/jninative.html
0 

Add a comment


-----------------------------------------------------------------------------------

Native Method example using JNI

Trivial JNI example
  1. Declare your native methods in an otherwise normal Java class.
    public class nativetest
    {
     public native String sayHello(String s);
     public static void main(String[] argv)
     {
      String retval = null;
      nativetest nt = new nativetest();
      retval = nt.sayHello("Beavis");
      System.out.println("Invocation returned " + retval);
     }
    }
    
  2. javac nativetest.java
    Standard Java compile
  3. javah -jni nativetest
    Generates .h files. Here is the code.
    /* DO NOT EDIT THIS FILE - it is machine generated */
    #include 
    /* Header for class nativetest */
    
    #ifndef _Included_nativetest
    #define _Included_nativetest
    #ifdef __cplusplus
    extern "C" {
    #endif
    /*
     * Class:     nativetest
     * Method:    sayHello
     * Signature: (Ljava/lang/String;)Ljava/lang/String;
     */
    JNIEXPORT jstring JNICALL Java_nativetest_sayHello
      (JNIEnv *, jobject, jstring);
    
    #ifdef __cplusplus
    }
    #endif
    #endif
    
  4. Write your native code, using the .h file generated above Copy the function prototype from the .h file and paste it in.
    #include "nativetest.h" /*double quotes tells it to search current directory*/
    
    JNIEXPORT jstring JNICALL Java_nativetest_sayHello
      (JNIEnv *env, jobject thisobject, jstring js)
    
    {
     return js;
    }
    
  5. Compile
    cl -c /Ic:\jdk1.1.6\include /Ic:\jdk1.1.6\include\win32 nativetest.c 
    link /libpath=c:\jdk1.1.6\lib nativetest.obj /dll
    
    The above uses Microsoft Visual C++ command-line tools.
  6. Modify your Java class to load the library.
    public class nativetest
    {
     static {
      System.loadLibrary("nativetest");
     }
     public native String sayHello(String s);
     public static void main(String[] argv)
     {
      String retval = null;
      nativetest nt = new nativetest();
      retval = nt.sayHello("Beavis");
      System.out.println("Invocation returned " + retval);
     }
    }
    
  7. run the example
    C:\jni\hello>java nativetest
    Invocation returned Beavis
    


    notes:

    2015年5月2日 星期六

    Mose code


    src:
    http://nomad1999.tripod.com/general/d015.htm

    content:



    通訊知識
    文, 表, 程式:  Blackmore

    摩斯電碼進階 
    (這一整頁有點大, 還請大家耐心些, 訓練程式在最下面)
      
    摩斯電碼說明
    1.摩斯電碼的組成英碼每組通常為五字
    例: VCDSG GTTUU POPOQ DDBNL GTRDY TTFUU DDRTY QROOP SXCGY GTTEJ
    中碼(數字碼)通常每組四字
    例:5548 9038 4830 8402 4850 5848 4830 47302 5849 4820
    2.摩斯電碼的聲響只有長短音的分別  " . "  中文練習讀音為"滴"  " - "中文練習讀音為"答"
    而電碼長短之時間如下:
    一個字 如 Z " --..
    " 的時間為一個單位  " " 的時間為三個單位 
    . "和 " - "的間隔時間為三個單位
     
    字和字 如 GR 為 --.  .-.        這時字和字間格的時間為三個單位
     
    字組和字組 GTRED  POQAZ  時間間格為五個時間單位
      
    詳細摩斯電碼表
    .  中文練習時念做"滴"       -  中文練習時念做"答"
    摩斯電碼及電話讀音法
    Letter摩斯電碼EnglishAmerican InternationalNATOAmerican (police)
    A.-AndrewAbelAmsterdamAlfaAdam
    B-...BenjaminBakerBaltimoreBravoBoy
    C-.-.CharlieCharlieCasablancaCharlieCharlie
    D-..DavidDogDenmarkDeltaDavid
    E.EdwardEasyEdisonEchoEdward
    F..-.FrederickFoxFloridaFoxtrotFrank
    G--.GeorgeGeorgeGallipoliGolfGeorge
    H....HarryHowHavanaHotelHenry
    I..IsaacItemItalyIndiaIda
    J.---JackJigJerusalemJulietJohn
    K-.-KingKingKilogramKiloKing
    L.-..LucyLoveLiverpoolLimaLincoln
    M--MaryMikeMadagascarMikeMary
    N-.NelliNanNew YorkNovemberNora
    O---OliverOboeOsloOscarOcean
    P.--.PeterPeterParisPapaPaul
    Q--.-QueenieQueenQuebecQuebecQueen
    R.-.RobertRogerRomaRomeoRobert
    S...SugarSugarSantiagoSierraSam
    T-TommyTareTripoliTangoTom
    U..-UncleUncleUppsalaUniformUnion
    V...-VictorVictorValenciaVictorVictor
    W.--WilliamWilliamWashingtonWhiskyWilliam
    X-..-XmasX (eks)XanthippeX-rayX-ray
    Y-.--YellowYokeYokohamaYankeeYoung
    Z--..ZebraZebraZürichZuluZebra
     
    數字及標點符號
    數字短碼長碼標點符號Code
    1.-.----句點 。.-.-.-
    2..-..---逗點 ,--..--
    3...--...--Colon---...
    4....-....-問號  ?..--..
    5..........撇號 ‵.----.
    6-....-....連字號  --....-
    7--...--...Fraction bar-..-.
    8-..---..括號(   )-.--.-
    9-.----.引號『 』.-..-.
    0------  
    訓練程式
    在上面的input欄輸入英文字母或文字,選擇速度後按下play即可
    英碼型式:FFKRP FKEOD SSKPQ XZLKF KOAQG KPLYO
    中碼型式:1245 6543 5533 6672 0954 3367 9707 1273 0836
    你也可以下載回去練習(內容包含練習表格,電報來去電紙)

    完成日期99-08-26, 最後修改日期99-08-26
    註: 本文是轉錄自Blackmore先生原放置於男人殺手站的文章
    此文獲得Blackmore先生同意轉載於此Nomad僅做校對與重新排版的工作
    在此特別再一次的感謝Blackmore 先生能授權本站轉載這部分的內容~~

    2015年3月17日 星期二

    EVT DVT PVT

    src:
    http://paul0306-home.blogspot.tw/2012/12/evtdvtpvt.html

    notes:
    - 15'3/17 : these 3 terms are easily confused.
    Actually, these term is close to HW validation including MFG (PVT for producible?).

    content:

    新產品在進行開發時,通常都會經過研發設計及工廠試投驗證階段,透過試產來取得樣品機,一方面是可以拿來作進一步的測試,也可以讓工廠盡早瞭解這個產品的製程並準備量產事宜,工廠也可以提出來自製造生產的意見(DFM, Design For Manufacturing),免得研發把東西設計出來,但卻出現工廠不能生產的情況。


    新產品在開發上一般會有三個試產的驗證階段,每個階段都可以有好幾次或是直接跳過,原則上應視實際需求而定,還有不要為了試產而試產,試產一定要有目的或必須性,否則試產既浪費錢又浪費時間。


    EVT : Engineering Verification Test (工程驗證測試階段)

    一般這個階段所生產出來的樣品只有電路板,研發工程師通常會先把他想要驗證的想法或是無法決定的設計擺在這種板子上進行驗證除錯使用

    如果研發的產品屬於全新的平台,第一次剛設計出來時,問題通常很多,到底要採取哪種可行的設計方案?所以有可能會有好幾次的 EVT修改

    這個階段的機構外殼如非使用舊有模具,便會使用mockup來進行驗證,主要目的是在模具發包製作前,用來驗證機構外殼設計是否符合需求,因為真正的模具費用很貴,所以要先驗證才能開模。

    如果設計是屬於修改既有的產品設計,那就會比較簡單,因為不會有太多的新技術,也就不需要太多的 EVT驗證,有時候我們也會視情況直接跳過 EVT而進入DVT。




    DVT: Design Verification Test (設計驗證測試階段)

    這是研發的第二階段,所有設計的驗證應該都已經完成了。這個時候會把機構的外殼加上來,另外電路板也要達到實際的尺寸大小,這樣才可以把電路板整個放到機構殼之中。

    這個階段要驗證整機的功能,重點是把設計及製造及安規的相關問題找出來並解決,以確保所有的設計都符合規格,而且可以導入生產。



    PVT: Production Verification Test(生產驗證測試階段)

    本階段的產品設計應該已經全部完成,設計驗證也告一段落。這個階段試產的目的是要做量產前的製造流程測試,所以必須要生產一定數量的產品,而且所有的生產程序都要符合製造廠的標準程序。

    另外還要確認所有的治工具、測試治具及生產設備數量是否可以符合量產的產能(capacity)需求,在這個階段工廠的PE人員就非常的重要,經由研發的手要把產品導入正式量產,這中間的工作一點都不輕鬆及時的對策及時和研發單位進行設計確認及反映、工廠各製程的文件撰寫及生產線人員及跟進都是一個新產品在導入時成功的關鍵。

    2015年3月3日 星期二

    [bug] make modules_prepare



    Symptom:
    - when compiling the driver, there maybe exist issue like :
    ......
    In file included from include/linux/gfp.h:4:0,
                     from include/linux/slab.h:12,
                     from include/linux/percpu.h:5,
                     from include/asm/local.h:4,
                     from include/linux/module.h:19,
                     from
    AX88179_178A_LINUX_DRIVER_v1.13.0_SOURCE/ax88179_178a.c:30:
    include/linux/mmzone.h:18:26: fatal error: linux/bounds.h: No such file or directory
     #include <linux/bounds.h>
                              ^
    .........

    Root Cause:
    - Header file miss.

    Solution:
    - go to kernel folder, issue the command:
    make modules_prepare 
    - after that, go back to make driver again

    ref:
    https://forums.gentoo.org/viewtopic-t-819002-start-0.html



    2015年3月1日 星期日

    [bug] Linux kernel menuconfig issue

    symptom:
    Makefile:434: *** mixed implicit and normal rules.  Stop.
    Makefile:1551: *** mixed implicit and normal rules.  Stop.

    root cause:
    old Makefile format

    solution:
    (434) config %config: scripts_basic outputmakefile FORCE
    (1551) / %/: prepare scripts FORCE

    -->
    (434) %config: scripts_basic outputmakefile FORCE
    (1551) %/: prepare scripts FORCE

    ref:
    http://www.cnblogs.com/jevan/archive/2011/03/20/1989199.html


    Linux online ebook

    src:
    http://www.linuxtopia.org/online_books/index.html

    notes:
    - 15'3/2:
      [1] this website lists some many Linux ebook including Andorid, iOS....etc. It's a great place for reference.
      [2] motiviation for this website is : kernel in a nutshell

    2015年2月26日 星期四

    no password for ssh

    src:
    - http://www.thegeekstuff.com/2008/11/3-steps-to-perform-ssh-login-without-password-using-ssh-keygen-ssh-copy-id/
     https://docs.google.com/file/d/0B543p5wfySTQRkw2VTNkcmtBcDg/edit (prtpdf)

    notes:
    - 15'2/26:
       [TODO] need to confirm whether it works?

    2015年2月25日 星期三

    Git introduction

    src:
    - http://git-scm.com/book/en/v2/Getting-Started-Git-Basics
      https://docs.google.com/file/d/0B543p5wfySTQY2l2c3JDbkw5U3M/edit (mypdf)


    notes:
    15'2/26:
    - Git use snapshot instead of file backup
    - offline support
    - 3 states : committed, modified, staged (loops)
    - [TODO] check the operation & setup


    2015年2月24日 星期二

    STONITH

    src:
    [S01] http://en.wikipedia.org/wiki/STONITH

    reference:
    [R01] http://yimu1023.blog.163.com/blog/static/362522822008621511567/

    content:
    - [S01]
    STONITH ("Shoot The Other Node In The Head" or "Shoot The Offending Node In The Head"), sometimes called STOMITH ("Shoot The Other Member/Machine In The Head"), is a technique for fencing in computer clusters.[1]
    Fencing is the isolation of a failed node so that it does not cause disruption to a computer cluster. As its name suggests, STONITH fences failed nodes by resetting or powering down the failed node.
    Multi-node error-prone contention in a cluster can have catastrophic results, such as if both nodes try writing to a shared storage resource. STONITH provides effective, if rather drastic, protection against these problems.
    Single node systems use a comparable mechanism called a watchdog timer. A watchdog timer will reset the node if the node does not tell the watchdog circuit that it is operating well. A STONITH decision can be based on various decisions which can be customer specific plugins



    notes:
    15'2/25:
    [S01]
    - an idea to prevent multi brain issue in a cluster. Need to check the further implementation [TODO]


    Multi-Volume Filesystem


    src:
    https://sort.symantec.com/public/documents/sfha/6.0/linux/productguides/html/sf_admin/ch25s01.htm

    content:

    About multi-volume file systems

    Veritas File System (VxFS) provides support for multi-volume file systems when used in conjunction with the Veritas Volume Manager. Using multi-volume support (MVS), a single file system can be created over multiple volumes, each volume having its own properties. For example, it is possible to place metadata on mirrored storage while placing file data on better-performing volume types such as RAID-1+0 (striped and mirrored). The volume must be in the same disk group as the volume set, and it cannot already be a member of another volume set.
    The MVS feature also allows file systems to reside on different classes of devices, so that a file system can be supported from both inexpensive disks and from expensive arrays. Using the MVS administrative interface, you can control which data goes on which volume types.
    Note:
    Multi-volume file system support is available only on file systems using disk layout Version 7 or later.


    notes:
    15'2/25:
    - a concept that meta & user data could be separated but not sure whether the user data could be
    (1) splitted and saved in different volume? [TODO]
    (2) duplication in different volume? [TODO]




    2015年1月28日 星期三

    Happy Path vs Exception Path(Alternate Path)

    src:
    http://en.wikipedia.org/wiki/Happy_path

    notes:
    - 15'1/28:
    Happy path is term which means a default path in flow description field. The contrast is Exception Path.





    2015年1月23日 星期五

    Buffer chip - 245

    src:
    http://www.datasheetarchive.com/74LS245%20buffer-datasheet.html

    notes:
    - 245 is a buffer chip w/ DIRECTION bit for GPIO direction control. It could be applied for PCH _SERIRQ



    2015年1月21日 星期三

    Synchronization, Backup, Replication, Consolidation


    Src:
    http://www.tgrmn.com/web/kb/item27.htm


    Notes:
    good explaination for these terms

    Synchronization : (SRC <- -> DST) src & dst are always synchronized
    Backup : (SRC -> DST ) Mirror, just from src to dst
    Replication: (SRC -> DST) Only update from src to remote, the old data will still keep in dst
    Consolidation : (SRC <- -> DST) src & dst will copy to each other once one file is missed in one side.





    2015年1月13日 星期二

    Btier


    src:
    http://sourceforge.net/projects/tier/
    http://www.lessfs.com/wordpress/
    http://searchstorage.techtarget.com/definition/Automated-storage-tiering-AST

    Relative term:
    Automated storage tiering (AST)


    notes:
    - a open source for storage tiering



    experiment for DRBD

    src:
    http://download.ithome.com.tw/article/index/id/2164

    note:
    - experiment for DRBD


    compare btw : RSYNC, DRBD, UNISON


    src:
    http://qq85609655.iteye.com/blog/1879416

    mypdf:
    https://drive.google.com/file/d/0B543p5wfySTQNmNyRVhjMFd6bHc/view?usp=sharing

    notes:
    - [TODO] need further check for features

    2015年1月7日 星期三

    multiple IP for single NIC

    src:
    http://xmodulo.com/how-to-assign-multiple-ip-addresses-to-one-network-interface-on-centos.html

    notes:

    ---------