Hadoop

Commetns:
- distributed architecture for compunting & storage
- could run in 3 modes : standalone , pseudo districuted, fully distributed
- client-server architect :
Server : run Name Node + Job Tracker
Client : run Data Node + Task Tracker
- the execution liverage counts on : MapReduce which is executed in slave side
-  HDFS is the filesystem used in hardoop, it will be kept in 3 different place. 2 is near-end side and the other is in far-end side
- HBASE is like a database could use for saving data after Reduce


Reference book:
- Windoop應用實作指南-掌握Hadoop翱翔雲端 by 許清榮,林奇暻, 買大誠


Thinking:
- if our traget data is distributed everywhere, maybe it's a way to handle locally and migrate.
- the point is map & key, the application must be divided & indexed via a key.

JNI coding

src:
http://blogforopensource.blogspot.tw/2012/08/jni-sample-java-code.html



notes:
15'5/5 :
- javah is used for JNI header
- pay attention to the c compile. It may need extra compiler like VC++




contents:

AUG

22

JNI sample Java Code

Blogging after a long time..


JNI - Java Native Interface.


Java interface to non­Java code.
It is Java's link to the "outside world"

•  Native methods are compiled into a dynamic link
  library (.dll, .so, etc.)
•  OS loads and links the library into the process
  that is running the Java Virtual Machine
•  Part of the Java Developer Kit(JDK),
  serves as a glue between Java side and
  native side of an application
• Allows Java code that runs inside a Java Virtual
  Machine (JVM) to inter operate with applications and
  libraries written in other programming languages,
  such as C, C++, and assembly

JNI w.r.t Operating System :

JNI Sequence:

If you are really interested in JNI,

You can find a Sample Code from here.

http://code.google.com/p/onlineexaminationsysteminjava/downloads/detail?name=samplejnicode.zip&can=2&q=

Reference:

http://patriot.net/~tvalesky/jninative.html

Posted 22nd August 2012 by Srikrishnan's Blog for open source

Labels: JNI sample Java Code

  

0 

Add a comment

-----------------------------------------------------------------------------------

Native Method example using JNI

Trivial JNI example

1. Declare your native methods in an otherwise normal Java class.

   public class nativetest
   {
    public native String sayHello(String s);
    public static void main(String[] argv)
    {
     String retval = null;
     nativetest nt = new nativetest();
     retval = nt.sayHello("Beavis");
     System.out.println("Invocation returned " + retval);
    }
   }
   

2. javac nativetest.java
   Standard Java compile
3. javah -jni nativetest
   Generates .h files. Here is the code.

   /* DO NOT EDIT THIS FILE - it is machine generated */
   #include 
   /* Header for class nativetest */
   
   #ifndef _Included_nativetest
   #define _Included_nativetest
   #ifdef __cplusplus
   extern "C" {
   #endif
   /*
    * Class:     nativetest
    * Method:    sayHello
    * Signature: (Ljava/lang/String;)Ljava/lang/String;
    */
   JNIEXPORT jstring JNICALL Java_nativetest_sayHello
     (JNIEnv *, jobject, jstring);
   
   #ifdef __cplusplus
   }
   #endif
   #endif
   

4. Write your native code, using the .h file generated above Copy the function prototype from the .h file and paste it in.

   #include "nativetest.h" /*double quotes tells it to search current directory*/
   
   JNIEXPORT jstring JNICALL Java_nativetest_sayHello
     (JNIEnv *env, jobject thisobject, jstring js)
   
   {
    return js;
   }
   

5. Compile

   cl -c /Ic:\jdk1.1.6\include /Ic:\jdk1.1.6\include\win32 nativetest.c 
   link /libpath=c:\jdk1.1.6\lib nativetest.obj /dll
   

   The above uses Microsoft Visual C++ command-line tools.
6. Modify your Java class to load the library.

   public class nativetest
   {
    static {
     System.loadLibrary("nativetest");
    }
    public native String sayHello(String s);
    public static void main(String[] argv)
    {
     String retval = null;
     nativetest nt = new nativetest();
     retval = nt.sayHello("Beavis");
     System.out.println("Invocation returned " + retval);
    }
   }
   

7. run the example

   C:\jni\hello>java nativetest
   Invocation returned Beavis
   

8. 
   

notes:

Mose code

src:
http://nomad1999.tripod.com/general/d015.htm

content:



通訊知識

文, 表, 程式:  Blackmore

---

摩斯電碼進階 

(這一整頁有點大, 還請大家耐心些, 訓練程式在最下面)

　 

摩斯電碼說明

1.摩斯電碼的組成英碼每組通常為五字

例: VCDSG GTTUU POPOQ DDBNL GTRDY TTFUU DDRTY QROOP SXCGY GTTEJ

中碼(數字碼)通常每組四字

例:5548 9038 4830 8402 4850 5848 4830 47302 5849 4820

2.摩斯電碼的聲響只有長短音的分別  " . "  中文練習讀音為"滴"  " - "中文練習讀音為"答"

而電碼長短之時間如下:

一個字 如 Z " --.." 

" . " 的時間為一個單位  " - " 的時間為三個單位 

" . "和 " - "的間隔時間為三個單位

　

字和字 如 GR 為 --.  .-.        這時字和字間格的時間為三個單位

　

字組和字組 GTRED  POQAZ  時間間格為五個時間單位

　 

詳細摩斯電碼表

.  中文練習時念做"滴"       -  中文練習時念做"答"

摩斯電碼及電話讀音法
Letter	摩斯電碼	English	American	International	NATO	American (police)
A	.-	Andrew	Abel	Amsterdam	Alfa	Adam
B	-...	Benjamin	Baker	Baltimore	Bravo	Boy
C	-.-.	Charlie	Charlie	Casablanca	Charlie	Charlie
D	-..	David	Dog	Denmark	Delta	David
E	.	Edward	Easy	Edison	Echo	Edward
F	..-.	Frederick	Fox	Florida	Foxtrot	Frank
G	--.	George	George	Gallipoli	Golf	George
H	....	Harry	How	Havana	Hotel	Henry
I	..	Isaac	Item	Italy	India	Ida
J	.---	Jack	Jig	Jerusalem	Juliet	John
K	-.-	King	King	Kilogram	Kilo	King
L	.-..	Lucy	Love	Liverpool	Lima	Lincoln
M	--	Mary	Mike	Madagascar	Mike	Mary
N	-.	Nelli	Nan	New York	November	Nora
O	---	Oliver	Oboe	Oslo	Oscar	Ocean
P	.--.	Peter	Peter	Paris	Papa	Paul
Q	--.-	Queenie	Queen	Quebec	Quebec	Queen
R	.-.	Robert	Roger	Roma	Romeo	Robert
S	...	Sugar	Sugar	Santiago	Sierra	Sam
T	-	Tommy	Tare	Tripoli	Tango	Tom
U	..-	Uncle	Uncle	Uppsala	Uniform	Union
V	...-	Victor	Victor	Valencia	Victor	Victor
W	.--	William	William	Washington	Whisky	William
X	-..-	Xmas	X (eks)	Xanthippe	X-ray	X-ray
Y	-.--	Yellow	Yoke	Yokohama	Yankee	Young
Z	--..	Zebra	Zebra	Zürich	Zulu	Zebra

　

數字及標點符號
數字	短碼	長碼	標點符號	Code
1	.-	.----	句點 。	.-.-.-
2	..-	..---	逗點 ，	--..--
3	...--	...--	Colon	---...
4	....-	....-	問號  ?	..--..
5	.....	.....	撇號 ‵	.----.
6	-....	-....	連字號  -	-....-
7	--...	--...	Fraction bar	-..-.
8	-..	---..	括號(   )	-.--.-
9	-.	----.	引號『　』	.-..-.
0	-	-----

訓練程式

在上面的input欄輸入英文字母或文字,選擇速度後按下play即可

英碼型式:FFKRP FKEOD SSKPQ XZLKF KOAQG KPLYO

中碼型式:1245 6543 5533 6672 0954 3367 9707 1273 0836

你也可以下載回去練習(內容包含練習表格,電報來去電紙)

---

完成日期99-08-26, 最後修改日期99-08-26

註: 本文是轉錄自Blackmore先生原放置於男人殺手站的文章

此文獲得Blackmore先生同意轉載於此, Nomad僅做校對與重新排版的工作

在此特別再一次的感謝Blackmore 先生能授權本站轉載這部分的內容～～

進入下一章

EVT DVT PVT

src:
http://paul0306-home.blogspot.tw/2012/12/evtdvtpvt.html

notes:
- 15'3/17 : these 3 terms are easily confused.
Actually, these term is close to HW validation including MFG (PVT for producible?).

content:

新產品研發驗證階段 (EVT/DVT/PVT)

新產品在進行開發時，通常都會經過研發設計及工廠試投驗證階段，透過試產來取得樣品機，一方面是可以拿來作進一步的測試，也可以讓工廠盡早瞭解這個產品的製程並準備量產事宜，工廠也可以提出來自製造生產的意見(DFM, Design For Manufacturing)，免得研發把東西設計出來，但卻出現工廠不能生產的情況。


新產品在開發上一般會有三個試產的驗證階段，每個階段都可以有好幾次或是直接跳過，原則上應視實際需求而定，還有不要為了試產而試產，試產一定要有目的或必須性，否則試產既浪費錢又浪費時間。

EVT : Engineering Verification Test (工程驗證測試階段)

一般這個階段所生產出來的樣品只有電路板，研發工程師通常會先把他想要驗證的想法或是無法決定的設計擺在這種板子上進行驗證除錯使用

如果研發的產品屬於全新的平台，第一次剛設計出來時，問題通常很多，到底要採取哪種可行的設計方案？所以有可能會有好幾次的 EVT修改

這個階段的機構外殼如非使用舊有模具，便會使用mockup來進行驗證，主要目的是在模具發包製作前，用來驗證機構外殼設計是否符合需求，因為真正的模具費用很貴，所以要先驗證才能開模。

如果設計是屬於修改既有的產品設計，那就會比較簡單，因為不會有太多的新技術，也就不需要太多的 EVT驗證，有時候我們也會視情況直接跳過 EVT而進入DVT。




DVT: Design Verification Test (設計驗證測試階段)

這是研發的第二階段，所有設計的驗證應該都已經完成了。這個時候會把機構的外殼加上來，另外電路板也要達到實際的尺寸大小，這樣才可以把電路板整個放到機構殼之中。

這個階段要驗證整機的功能，重點是把設計及製造及安規的相關問題找出來並解決，以確保所有的設計都符合規格，而且可以導入生產。



PVT: Production Verification Test(生產驗證測試階段)

本階段的產品設計應該已經全部完成，設計驗證也告一段落。這個階段試產的目的是要做量產前的製造流程測試，所以必須要生產一定數量的產品，而且所有的生產程序都要符合製造廠的標準程序。

另外還要確認所有的治工具、測試治具及生產設備數量是否可以符合量產的產能(capacity)需求，在這個階段工廠的PE人員就非常的重要，經由研發的手要把產品導入正式量產，這中間的工作一點都不輕鬆及時的對策及時和研發單位進行設計確認及反映、工廠各製程的文件撰寫及生產線人員及跟進都是一個新產品在導入時成功的關鍵。

[bug] make modules_prepare

Symptom:
- when compiling the driver, there maybe exist issue like :
......
In file included from include/linux/gfp.h:4:0,
                 from include/linux/slab.h:12,
                 from include/linux/percpu.h:5,
                 from include/asm/local.h:4,
                 from include/linux/module.h:19,
                 from
AX88179_178A_LINUX_DRIVER_v1.13.0_SOURCE/ax88179_178a.c:30:
include/linux/mmzone.h:18:26: fatal error: linux/bounds.h: No such file or directory
 #include <linux/bounds.h>
                          ^

.........

Root Cause:

- Header file miss.

Solution:

- go to kernel folder, issue the command:

make modules_prepare 

- after that, go back to make driver again

ref:
https://forums.gentoo.org/viewtopic-t-819002-start-0.html

[bug] Linux kernel menuconfig issue

symptom:
Makefile:434: *** mixed implicit and normal rules.  Stop.
Makefile:1551: *** mixed implicit and normal rules.  Stop.

root cause:
old Makefile format

solution:
(434) config %config: scripts_basic outputmakefile FORCE
(1551) / %/: prepare scripts FORCE

-->
(434) %config: scripts_basic outputmakefile FORCE
(1551) %/: prepare scripts FORCE

ref:
http://www.cnblogs.com/jevan/archive/2011/03/20/1989199.html

Linux online ebook

src:
http://www.linuxtopia.org/online_books/index.html

notes:
- 15'3/2:
  [1] this website lists some many Linux ebook including Andorid, iOS....etc. It's a great place for reference.
  [2] motiviation for this website is : kernel in a nutshell

no password for ssh

src:
- http://www.thegeekstuff.com/2008/11/3-steps-to-perform-ssh-login-without-password-using-ssh-keygen-ssh-copy-id/
 https://docs.google.com/file/d/0B543p5wfySTQRkw2VTNkcmtBcDg/edit (prtpdf)

notes:
- 15'2/26:
   [TODO] need to confirm whether it works?

Git introduction

src:
- http://git-scm.com/book/en/v2/Getting-Started-Git-Basics
  https://docs.google.com/file/d/0B543p5wfySTQY2l2c3JDbkw5U3M/edit (mypdf)


notes:
15'2/26:
- Git use snapshot instead of file backup
- offline support
- 3 states : committed, modified, staged (loops)
- [TODO] check the operation & setup

STONITH

src:
[S01] http://en.wikipedia.org/wiki/STONITH

reference:
[R01] http://yimu1023.blog.163.com/blog/static/362522822008621511567/

content:
- [S01]
STONITH ("Shoot The Other Node In The Head" or "Shoot The Offending Node In The Head"), sometimes called STOMITH ("Shoot The Other Member/Machine In The Head"), is a technique for fencing in computer clusters.^[1]

Fencing is the isolation of a failed node so that it does not cause disruption to a computer cluster. As its name suggests, STONITH fences failed nodes by resetting or powering down the failed node.

Multi-node error-prone contention in a cluster can have catastrophic results, such as if both nodes try writing to a shared storage resource. STONITH provides effective, if rather drastic, protection against these problems.

Single node systems use a comparable mechanism called a watchdog timer. A watchdog timer will reset the node if the node does not tell the watchdog circuit that it is operating well. A STONITH decision can be based on various decisions which can be customer specific plugins

notes:
15'2/25:
[S01]
- an idea to prevent multi brain issue in a cluster. Need to check the further implementation [TODO]

Multi-Volume Filesystem

src:
https://sort.symantec.com/public/documents/sfha/6.0/linux/productguides/html/sf_admin/ch25s01.htm

content:

About multi-volume file systems

Veritas File System (VxFS) provides support for multi-volume file systems when used in conjunction with the Veritas Volume Manager. Using multi-volume support (MVS), a single file system can be created over multiple volumes, each volume having its own properties. For example, it is possible to place metadata on mirrored storage while placing file data on better-performing volume types such as RAID-1+0 (striped and mirrored). The volume must be in the same disk group as the volume set, and it cannot already be a member of another volume set.

The MVS feature also allows file systems to reside on different classes of devices, so that a file system can be supported from both inexpensive disks and from expensive arrays. Using the MVS administrative interface, you can control which data goes on which volume types.

Note:

Multi-volume file system support is available only on file systems using disk layout Version 7 or later.

notes:
15'2/25:
- a concept that meta & user data could be separated but not sure whether the user data could be
(1) splitted and saved in different volume? [TODO]
(2) duplication in different volume? [TODO]

Happy Path vs Exception Path(Alternate Path)

src:
http://en.wikipedia.org/wiki/Happy_path

notes:
- 15'1/28:
Happy path is term which means a default path in flow description field. The contrast is Exception Path.

Buffer chip - 245

src:
http://www.datasheetarchive.com/74LS245%20buffer-datasheet.html

notes:
- 245 is a buffer chip w/ DIRECTION bit for GPIO direction control. It could be applied for PCH _SERIRQ

Synchronization, Backup, Replication, Consolidation

Src:
http://www.tgrmn.com/web/kb/item27.htm


Notes:
good explaination for these terms

Synchronization : (SRC <- -> DST) src & dst are always synchronized
Backup : (SRC -> DST ) Mirror, just from src to dst
Replication: (SRC -> DST) Only update from src to remote, the old data will still keep in dst
Consolidation : (SRC <- -> DST) src & dst will copy to each other once one file is missed in one side.

Btier

src:
http://sourceforge.net/projects/tier/
http://www.lessfs.com/wordpress/
http://searchstorage.techtarget.com/definition/Automated-storage-tiering-AST

Relative term:
Automated storage tiering (AST)


notes:
- a open source for storage tiering

experiment for DRBD

src:
http://download.ithome.com.tw/article/index/id/2164

note:
- experiment for DRBD

compare btw : RSYNC, DRBD, UNISON

src:
http://qq85609655.iteye.com/blog/1879416

mypdf:
https://drive.google.com/file/d/0B543p5wfySTQNmNyRVhjMFd6bHc/view?usp=sharing

notes:
- [TODO] need further check for features

multiple IP for single NIC

src:
http://xmodulo.com/how-to-assign-multiple-ip-addresses-to-one-network-interface-on-centos.html

notes:

---------