Internship Programs

Dear Students,

kindly click on the link below it’s a important news for all the engineering students –

http://www.newsworldindia.in/india/3-internships-compulsory-for-engineering-students-from-this-year-mhrd/269530/

Do not worry if you have not yet planned anything for your internship, WOIR Software has brought a golden chance for you. 

Get yourself enrolled today by sending email at info@woir.in in for various latest industrial skill programs.

Checkout  http://woir.in/big-data-analytics-training-internship for more details.

 

 For any queries talk to our Program Manager : +917500000851

Thank you

Regards

WOIR Software Pvt.Ltd

Big Data Analytics Training & Internship

WOIR Software Announces Internship PLUS training on Big Data Analytics

  • Mentorship under highly experienced IITian
  • Tenure approx. 6 to 8 weeks or more depending on your need, full time/part time ( 11:30 am to 8:30 pm )
  • Limited seats, small batches for trainings and internships
  • The position/training is based in Hyderabad (Near Hitex City)
  • Training will include following latest technologies(contact us for more detail)
    • Hadoop, HDFS, Map Reduce, Hive, PIG
    • NoSQL DBs, Ananlytics ( Elasticsearch, Logstash Kibana )
    • ETL, Message Brokers, End to End Pipeline
  • Internship will include LIVE Project Hands-on Experience
    • The work will include coding in python, shell, web based application and Big Data, NoSQL, JAVA, Angular – depending on candidates skillset
    • Necessary tools and industry standard oriented workflow
    • Version Control System GIT, Agile Methodology using JIRA
    • Internship + Training Certificate by a Pvt. Ltd. Company

Please understand that this is live project position, it will require aggressive working schedule – don’t apply if you are not willing to learn and live in the hard environment.

Contact @ +91-7500000851 for any query.

Please write email to info@woir.in with your resume along with the reference (how you did you get to know about this position) if you want to apply for the position.

You can also apply directly through – http://woir.in/analytics/register.html ( select WOIR tab ).

WOIR is Hiring Interns/Trainees from 3rd Week of March, 2018

WOIR Software is looking for interns/trainees. Hiring will include interview where he/she will be asked to demonstrate the logical thinking and programming skill.It is unpaid position, they would be given a valid certificate from private limited company for the period.

  • Tenure approx. 3 months, full time
  • The position is based in Hyderabad (near Hitex City)
  • The work will include coding in python, shell, web based application and Big Data, NoSQL, JAVA, Angular – depending on candidates skillset
  • He/she should be excellent in programming, logical thinking, data structures
  • Should be willing to learn new technologies and should be excited to work in a startup
  • Fresh Passout/B. Tech 3rd and 4th year students can apply

Please write email to info@woir.in with your resume along with the reference (how you did you get to know about this position) if you want to apply for the position.

You can also apply directly through – http://woir.in/analytics/register.html ( select WOIR tab ).

nginx

sudo apt-get install nginx

sudo service nginx stop

sudo nginx -c $PWD/nginx_keep_alive.conf

######### To run Nginx as a “100% transparent” proxy for Elasticsearch ###############

events {
    worker_connections  1024;
}
http {
  server {
    listen 8080;
    location / {
      proxy_pass http://localhost:9200;
    }
  }
}



######################## Persistent Connection ##########################

events {
    worker_connections  1024;
}

http {

  upstream elasticsearch {
    server localhost:9200;

    keepalive 15;
  }

  server {
    listen 8080;

    location / {
      proxy_pass http://elasticsearch;
      proxy_http_version 1.1;
      proxy_set_header Connection "Keep-Alive";
      proxy_set_header Proxy-Connection "Keep-Alive";
    }

  }

}

################### Load Balancer - Round Robin Fashion ##########################




events {
    worker_connections  1024;
}

http {

  upstream elasticsearch {
    server 127.0.0.1:9200;
    server 127.0.0.1:9201;
    server 127.0.0.1:9202;

    keepalive 15;
  }

  server {
    listen 8080;

    location / {
      proxy_pass http://elasticsearch;
      proxy_http_version 1.1;
      proxy_set_header Connection "Keep-Alive";
      proxy_set_header Proxy-Connection "Keep-Alive";
    }

  }

}

Logstash

# Source the following file to setup the java version correctly
    source ~/sourceme (to setup the java version correctly)

# Logstash to try from command line
    logstash -e 'input { stdin { } } output { stdout {} }'


# Logstash to try from command line
    logstash -e 'input { stdin { } } output { stdout { codec => rubydebug } }'

    logstash -e 'input { stdin { } } output { elasticsearch { hosts => localhost } }'


# Check if Logstash has been able to insert data into the diretory
    curl -XGET "http://localhost:9200/_cat/indices?pretty

# Run
    logstash -e 'input { stdin { } } output { elasticsearch { hosts => localhost } }'

# List the indices 
    curl -XGET "http://localhost:9200/_cat/indices?pretty"

# After it is started enter “Hello Your Name”



# Run -	
    curl -XGET "http://localhost:9200/logstash-*/_search"?pretty


# Download sample log file -
    cd /home/woir/Downloads/ && wget https://download.elastic.co/demos/logstash/gettingstarted/logstash-tutorial.log.gz
    gunzip logstash-tutorial.log.gz


# Save  following file in your home directory as apache.conf


########## apache.conf ##############
input {
    file {
        path => "/home/woir/Downloads/logstash-tutorial.log"
        start_position => beginning
        sincedb_path => "/dev/null"
        ignore_older => 0
    }
}

filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
    date {
        match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
    }
    geoip {
        source => "clientip"
    }
}

output {
    elasticsearch {
        index=>"gabbar"
    }
}

# Start pipeline 
    logstash -f ~/apache.conf


# See the contents –
    curl -XGET http://localhost:9200/logstash-2015.01.04/_search




######## Apple Stocks Hands on session  ####### 


1. Download the data to be inserted into ES
    wget -O /home/woir/Downloads/table-3.csv http://woir.in/wp-content/uploads/2016/12/table-3.csv





## save the following file as /home/woir/apple.conf
input {
  file {
    path => "/home/woir/Downloads/table*.csv"
    type => "core2"
    start_position => "beginning"    
  }
}
filter {
  csv {
      separator => ","
      columns => ["Date","Open","High","Low","Close","Volume","Adj Close"]
  }
  mutate {convert => ["High", "float"]}
  mutate {convert => ["Open", "float"]}
  mutate {convert => ["Low", "float"]}
  mutate {convert => ["Close", "float"]}
  mutate {convert => ["Volume", "float"]}
}
output {  
    elasticsearch {
        action => "index"
        hosts => ["localhost:9200"]
        index => "stock"
        workers => 1
    }
    stdout {}
}

# Point the config file and run the logstash – it will insert data into elasticsearch
    /home/woir/logstash-5.0.2/bin/logstash -f  /home/woir/apple.conf

# Check data insertion is done or not –
    curl -XGET http://localhost:9200/stock/_search?pretty

Elasticsearch Queries

### Run from your terminal 
## it tells if elasticsearch is up or not
### 
GET /

### Create Index
PUT /govtcompany

### Get Status
GET /govtcompany1?pretty

### Get List of Indices 
GET /_cat/indices?pretty&v

GET /_cat/indices/com*?v&s=index


### Delete and Index-
DELETE /govtcompany
GET /_cat/indices?pretty

### Create index named “company”
PUT /company

### Close Index
POST /company/_close
GET /_cat/indices?pretty

#### a simple search/index on the index which is closed will lead to the exception

GET /company/_search

POST /company/_open
 
######### Index not found exception
POST /myindex1/_close
POST /myindex1/_open

### Open Index
POST /company/_open
GET /_cat/indices?pretty

### Aliases
GET /_cat/aliases?v

DELETE /test1
DELETE /test2
DELETE /test3
DELETE /test4


### Please Create Index test1,test2, test3, test4
PUT /test1
PUT /test2
PUT /test3
PUT /test4
GET /_cat/indices
GET /_cat/aliases

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "test1",
        "alias": "alias1"
      }
    }
  ]
}

GET /_cat/aliases

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "test1",
        "alias": "alias2"
      }
    }
  ]
}

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "test3",
        "alias": "alias1"
      }
    }
  ]
}

POST /_aliases
{
  "actions": [
    {
      "remove": {
        "index": "test1",
        "alias": "alias1"
      }
    }
  ]
}

POST /_aliases
{
  "actions": [
    {
      "add": {
        "indices": [
          "test1",
          "test2"
        ],
        "alias": "alias1"
      }
    }
  ]
}
######### OK use regular expression
POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "test*",
        "alias": "alias1"
      }
    }
  ]
}


POST /_aliases
{
  "actions": [
    {
      "remove": {
        "index": "test1",
        "alias": "alias1"
      }
    }
  ]
}

######## use regular expression
POST /_aliases
{
  "actions": [
    {
      "remove": {
        "index": "test*",
        "alias": "alias1"
      }
    }
  ]
}

######## will come back to this later - very intereting scenario here
### Filter Alias 
PUT /test1
{
  "mappings": {
    "type1": {
      "properties": {
        "user": {
          "type": "keyword"
        }
      }
    }
  }
}

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "test1",
        "alias": "alias2",
        "filter": {
          "term": {
            "user": "kimchy"
          }
        }
      }
    }
  ]
}

DELETE /test1

PUT /test1
{
  "mappings": {
    "type1": {
      "properties": {
        "user": {
          "type": "keyword"
        }
      }
    }
  }
}

POST /test1/type1
{
  "user": "Amar Sharma1",
  "age": 45,
  "experience": 10
}

POST /test1/type1
{
  "user": "Amar Sharma2",
  "age": 45,
  "experience": 10
}

POST /test1/type1
{
  "user": "Amar Sharma3",
  "age": 45,
  "experience": 10
}

POST /test1/type1
{
  "user": "kimchy",
  "age": 45,
  "experience": 10
}

POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "test1",
        "alias": "alias2",
        "filter": {
          "term": {
            "user": "kimchy"
          }
        }
      }
    }
  ]
}

GET /alias2/_search

### Data Insertion#########
###### schemaless
POST /company/employee
{
  "name": "Amar Sharma",
  "age": 45,
  "experience": 10
}


###### record recovery using id

GET company/employee/AV_ksw2CXaNB5r6xnSki


### Routing
PUT /my_index

PUT /my_index/my_type/1?routing=user1&refresh=true&pretty
{
  "title": "This is a document"
}

GET /my_index?pretty


PUT /my_index/my_type/1?routing=user2&refresh=true&pretty
{
  "title": "This is a document"
}

GET /my_index/my_type/1?routing=user1&pretty
GET /my_index/my_type/1?&pretty

PUT /test1/type1/1?routing=user1&refresh=true&pretty
{
  "user": "Srinivas1"
}

PUT /test1/type1/1?routing=user1&refresh=true&pretty
{
  "user": "Srinivas"
}

GET /test1/type1/1

GET /test1/type1/1?routing=amar

GET /test1/type1/1?routing=user3

DELETE company

PUT /company
{
  "mappings": {
    "employee2": {
      "properties": {
        "age": {
          "type": "long"
        },
        "experience": {
          "type": "long"
        },
        "name": {
          "type": "string",
          "analyzer": "standard"
        }
      }
    }
    
  }
}

DELETE company

GET /company?pretty

### Create Mappings ( multiple )
PUT /company
{
  "mappings": {
    "employee": {
      "properties": {
        "age": {
          "type": "long"
        },
        "experience": {
          "type": "long"
        },
        "name": {
          "type": "string",
          "analyzer": "standard"
        }
      }
    },    
    "staff": {
      "properties": {
        "age": {
          "type": "long"
        },
        "experience": {
          "type": "long"
        },
        "name": {
          "type": "string",
          "analyzer": "standard"
        }
      }
    }
    
  }
}

GET /company
### Get the mappings
GET /company?pretty

########### Data Insertion 

PUT /company1
{
  "mappings": {
    "employee": {
      "properties": {
        "age": {
          "type": "long"
        },
        "experience": {
          "type": "long"
        },
        "name": {
          "type": "string",
          "analyzer": "standard"
        }
      }
    }
  }
}

GET /company1/employee/_mapping

GET /company/_search

GET /company1/_search

DELETE /company

PUT /company
{
  "mappings": {
    "employee": {
      "properties": {
        "age": {
          "type": "long"
        },
        "experience": {
          "type": "long"
        },
        "name": {
          "type": "string",
          "analyzer": "standard"
        }
      }
    }
  }
}

POST /company/employee
{
  "name": "Amar Sharma",
  "age": 45,
  "experience": 10
}

POST /company/employee
{
  "name": "Sriknaht Kandi",
  "age": 35,
  "experience": 7
}

POST /company/employee
{
  "name": "Abdul Malik",
  "age": 25,
  "experience": 3
}

######## to specific ID ####


####### auto generated id
POST /company/employee/
{
  "name": "Amar3 Sharma",
  "age": 45,
  "experience": 10
}


POST /company/employee/2
{
  "name": "Amar3 Sharma",
  "age": 45,
  "experience": 10
}

##### let me repeat the above one and notie the version
POST /company/employee/2
{
  "name": "Amar3 Sharma",
  "age": 45,
  "experience": 10
}



GET /company/employee/2


POST /company/employee/2
{
  "name": "Amar3 Sharma",
  "age": 45
}

PUT /company/employee/2
{
  "name": "Anant Sharma",
  "age": 45,
  "FatherName": "Amar Sharma"
}

PUT /company/employee/3
{
  "name": "Anant Sharma1",
  "age": 45,
  "FatherName": "Amar Sharma"
}

PUT /company/employee/4
{
  "name": "Amar5 Sharma",
  "age": 45,
  "FatherName": "Suresh Chandra"
}

PUT /company1/employee/6
{
  "name": "Amar6 Sharma6",
  "age": 45,
  "FatherName": "Suresh Chandra"
}

###### in place updates #######

POST /company/employee/2/_update
{
  "script": "ctx._source.name='Anant2'",
  "retry_on_conflict": 3
}

###########Analyzer

POST /_analyze
{
  "analyzer": "whitespace",
  "text":     "The quick brown fox!"
}

POST /_analyze
{
  "analyzer": "standard",
  "text":     "The quick brown fox."
}

POST /_analyze
{
  "tokenizer": "standard",
  "filter":  [ "lowercase", "asciifolding" ],
  "text":      "Is this déja vu? A quick fox jump over lazy pumped"
}

POST /_analyze
{
  "tokenizer": "standard",
  "filter":  [ "lowercase", "asciifolding","stemmer" ],
  "text":      "Is this déja vu? A quicks foxes jumped over lazy dog"
}

POST /_analyze
{
  "tokenizer": "standard",
  "filter":  [ "synonym", "asciifolding" ],
  "text":      "Is this déja vu?"
}
POST _analyze
{
  "analyzer": "simple",
  "text":     "The 5 quic4k brown fox."
}

POST /_analyze
{
  "analyzer": "simple",
  "text":     "Time2School."
}

##########SEARCH

GET /company/employee/_search?pretty

GET /company*/staff,employee/_search?pretty

GET /_search
{
  "query": {
    "match_all": {}
  }
}

GET /company/employee/_search?pretty=true
{
  "query": {
    "query_string": {
      "query": "Amar"
    }
  }
}

GET /_search?pretty
{
  "query": {
    "match": {
      "name": "Amar Sharma"
    }
  }
}

GET /company/employee/_search?pretty=true
{
  "query": {
    "query_string": {
      "query": "Amar*"
    }
  }
}

########## careful, now filtered is no more supported
GET /company/employee/_search?pretty=true
{
  "query": {
    "filtered": {
      "filter": {
        "term": {
          "name": "Amar"
        }
      }
    }
  }
}

GET /company/employee/_search?pretty
{
  "query": {
    "range": {
      "age": {
        "from": 15,
        "to": 50
      }
    }
  }
}

GET /company/employee/_search?pretty
{
  "query": {
    "range": {
      "age": {
        "from": 26,
        "to": 50
      }
    }
  }
}

GET /company/employee/_search?pretty
{
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "name": "kandi"
          }
        },
        {
          "match": {
            "name": "Anant Sharma"
          }
        }
      ],
      "must": {
        "range": {
          "age": {
            "from": 45,
            "to": 55
          }
        }
      },
      "minimum_should_match": 2
    }
  }
}

GET /_search?pretty
{
  "query": {
    "bool": {
      "must": {
        "match": {
          "name": "Anant"
        }
      },
      "should": {
        "range": {
          "age": {
            "gte": 46
          }
        }
      }
    }
  }
}


GET /_search?pretty=true
{
  "query": {
    "match": {
      "name": {
        "query": "Anant Sharma",
        "operator": "or"
      }
    }
  }
}


GET /_search?pretty=true
{
  "query": {
    "match": {
      "name": {
        "query": "Anant Sharma",
        "operator": "and"
      }
    }
  }
}

POST /company/employee
{
  "name": "Abdul8 Malik8",
  "username": "abdul2013",
  "age": 25,
  "experience": 3,
  "date": "2013-02-01",
  "FatherName" : "@Amar"
}

POST /company/employee
{
  "name": "Abdul9 Malik9",
  "username": "abdul 2013",
  "age": 25,
  "experience": 3,
  "date": "2014-02-01",
  "FatherName" : "@Amar"
}

POST /_search
{
  "query": {
    "term" : { "username" : "abdul2013" } 
  }
}

POST /_search
{
  "query": {
    "term" : { "username" : "abdul 2013" } 
  }
}

GET /company/_search?pretty=true
{
  "query": {
    "bool": {
      "should": [
        {
          "term": {
            "username": "Amar"
          }
        },
        {
          "term": {
            "username": "abdul2013"
          }
        }
      ],
      "minimum_should_match": 0

    }
  }
}

GET /company/_search 
GET /company,company1/_search 
GET /company*/_search 
GET /company/employee/_search 
GET /company/staff,employee/_search 
GET /company/employee/_search 
GET /_all/employee/_search 

GET /company/_search?size=5&from=0 
GET /company/_search?size=5&from=5 
GET /company/_search?size=5&from=10 

GET /_search?q=name:Sharma
GET /_search?q=name:Amar5
GET /_search?q=name:Amar

POST /company/employee
{
  "name": "Abdul2 Malik2",
  "age": 25,
  "experience": 3,
  "date": "2013-04-01"
}
POST /company/employee
{
  "name": "Abdul3 Malik3",
  "age": 25,
  "experience": 3,
  "date": "2013-03-01"
}

POST /company/employee
{
  "name": "Abdul4 Malik4",
  "age": 25,
  "experience": 3,
  "date": "2013-02-01"
}

GET /_search?q=+date:>2013-03-01+name:Amar

POST /company/employee
{
  "name": "Abdul4 Malik4",
  "age": 25,
  "experience": 3,
  "date": "2013-02-01",
  "FatherName" : "@Amar"
}

######## are same #######
GET /_search?q=_all:Amar
GET /_search?q=Amar

############################

POST /company/employee
{
  "name": "Abdul7 Malik7",
  "username": "abdul2013",
  "age": 25,
  "experience": 3,
  "date": "2014-02-01",
  "FatherName" : "@Amar"
}

POST /company/employee
{
  "name": "Abdul7 Malik7",
  "username": "abdul2013",
  "age": 25,
  "experience": 3,
  "date": "2014-02-01",
  "FatherName" : "@Amar"
}

POST /company/employee
{
  "name": "Abdul8 Malik8",
  "username": "abdul2013",
  "age": 25,
  "experience": 3,
  "date": "2013-02-01",
  "FatherName" : "@Amar"
}

POST /company/employee
{
  "name": "Abdul9 Malik9",
  "username": "abdul 2013",
  "age": 25,
  "experience": 3,
  "date": "2014-02-01",
  "FatherName" : "@Amar"
}

GET /company/_mapping

GET /company/_search?q=2013

GET /company/_search?q=date:2013

POST /company/employee
{
  "name": "Andrew Thomas",
  "username": "Abdul9 Malik9",
  "age": 25,
  "experience": 3,
  "date": "2014-02-01",
  "FatherName" : "David Knol"
}

GET /_search?pretty
{
  "query": {
    "bool": {
      "must": {
        "match": {
          "name": "Andrew Gabbar"
        }
      },
      "should": {
        "range": {
          "age": {
            "gte": 15
          }
        }
      }
    }
  }
}




POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "myindex",
        "alias": "alias1"
      }
    }
  ]
}

PUT /myapp
{
  "mapping": {
    "tweet": {
      "properties": {
        "tweet": {
          "type": "string"
        },
        "name": {
          "type": "string"
        },
        "nick": {
          "type": "string"
        },
        "Date": {
          "type": "date", "format": "yyyy-MM-dd HH:mm:ss"
        },
        "rt": {
          "type": "long"
        }
      }
    }
  }
}

POST _analyze
{
  "analyzer": "whitespace",
  "text":     "The Brown-Cow's Part_No. #A.BC123-456 joe@bloggs.com"
}

Synopsys




GET _search
{
 "query": {
 "match_all": {}
 }
}

POST /_aliases
{
 "actions": [
 {
 "add": {
 "index": "myindex",
 "alias": "alias1"
 }
 }
 ]
}

DELETE /test1

PUT /test1
{
 "mappings": {
 "type1": {
 "properties": {
 "user": {
 "type": "keyword"
 }
 }
 }
 }
}

POST /test1/type1
{
 "user": "Amar Sharma1",
 "age": 45,
 "experience": 10
}


POST /test1/type1
{
 "user": "Amar Sharma2",
 "age": 45,
 "experience": 10
}

POST /test1/type1
{
 "user": "Amar Sharma3",
 "age": 45,
 "experience": 10
}

POST /test1/type1
{
 "user": "kimchy",
 "age": 45,
 "experience": 10
}

POST /_aliases
{
 "actions": [
 {
 "add": {
 "index": "test1",
 "alias": "alias2",
 "filter": {
 "term": {
 "user": "kimchy"
 }
 }
 }
 }
 ]
}

GET /alias2/_search

PUT /test1/type1/1?routing=user1&refresh=true&pretty
{
 "user": "Srinivas"
}

PUT /test1/type1/1?routing=user1&refresh=true&pretty
{
 "user": "Srinivas"
}

GET /test1/type1/1

GET /test1/type1/1?routing=amar

GET /test1/type1/1?routing=user1


PUT /company1
{
 "mappings": {
 "employee": {
 "properties": {
 "age": {
 "type": "long"
 },
 "experience": {
 "type": "long"
 },
 "name": {
 "type": "string",
 "analyzer": "standard"
 }
 }
 }
 }
}

GET company1/employee/_mapping

GET company/_search

GET company1/_search

DELETE company

PUT /company
{
 "mappings": {
 "employee": {
 "properties": {
 "age": {
 "type": "long"
 },
 "experience": {
 "type": "long"
 },
 "name": {
 "type": "string",
 "analyzer": "standard"
 }
 }
 }
 }
}
 
POST /company/employee
{
 "name": "Amar Sharma",
 "age": 45,
 "experience": 10
}

POST /company/employee
{
 "name": "Sriknaht Kandi",
 "age" : 35,
 "experience" : 7
 }

POST /company/employee
{
 "name": "Abdul Malik",
 "age" : 25,
 "experience" : 3,
 }

PUT /company/employee/2
{
 "name": "Amar3 Sharma",
 "age" : 45,
 "experience" : 10
 }
GET /company/employee/2


PUT /company/employee/2
{
 "name": "Amar3 Sharma",
 "age" : 45
 }

PUT /company/employee/2
{
 "name": "Anant Sharma",
 "age" : 45,
 "FatherName": "Amar Sharma"
 }
PUT /company/employee/3
{
 "name": "Anant Sharma1",
 "age" : 45,
 "FatherName": "Amar Sharma"
 }
 
PUT /company/employee/4
{
 "name": "Amar5 Sharma",
 "age" : 45,
 "FatherName": "Suresh Chandra"
 }
 
PUT /company1/employee/6
{
 "name": "Amar6 Sharma6",
 "age" : 45,
 "FatherName": "Suresh Chandra"
 }
 
 
POST /company/employee/2/_update
{
"script": "ctx._source.age+=1", 
"retry_on_conflict": 3
}

GET /company/employee/_search?pretty

GET /company/employee/_search?pretty=true
{ 
 "query": {
 "query_string": {
 "query": "Amar"
 }
 }
}

GET /_search?pretty
{
 "query": {
 "match": {
 "name": "Amar Sharma"
 }
 }
 }

GET /company/employee/_search?pretty=true
{ 
 "query": {
 "query_string": {
 "query": "Amar*"
 }
 }
}
 
GET /test-data/cities/_search?pretty=true
{
 "query": {
 "filtered": {
 "filter": {
 "term": {
 "name": "Amar"
 }
 }
 }
 }
}

GET /company/employee/_search?pretty
{
 "query": {
 "range": {
 "age": {
 "from": 15,
 "to": 50
 }
 }
 }
}
 
GET /company/employee/_search?pretty
{
 "query": {
 "range": {
 "age": {
 "from": 26,
 "to": 50
 }
 }
 }
}


GET /company/employee/_search?pretty
{
 "query": {
 "bool": {
 "should": [
 {
 "match": {
 "name": "kandi1"
 }
 },
 {
 "match": {
 "name": "Anant Sharma"
 }
 }
 ],
 "must": {
 "range": {
 "age": {
 "from": 45,
 "to": 55
 }
 }
 },
 "minimum_should_match": 1
 }
 }
}


GET /_search?pretty
{
 "query": {
 "bool": {
 "must": {
 "match": {
 "name": "Anant"
 }
 },
 "should": {
 "range": {
 "age": {
 "gte": 46
 }
 }
 }
 }
 }
}


GET /index/_search 
GET /index1,index2/_search 
GET /company*/_search 
GET /index/type/_search 
GET /index/type1,type2/_search 
GET /index/type*/_search 
GET /_all/type*/_search


GET /company/_search?size=5&from=0 
GET /company/_search?size=5&from=5 
GET /company/_search?size=5&from=10

GET /_search?q=name:Sharma
GET /_search?q=name:Amar5
GET /_search?q=name:Amar

POST /company/employee
{
 "name": "Abdul2 Malik2",
 "age": 25,
 "experience": 3,
 "date": "2013-04-01"
}
POST /company/employee
{
 "name": "Abdul3 Malik3",
 "age": 25,
 "experience": 3,
 "date": "2013-03-01"
}

POST /company/employee
{
 "name": "Abdul4 Malik4",
 "age": 25,
 "experience": 3,
 "date": "2013-02-01"
}

GET /_search?q=+date:>2013-03-01+name:Amar

POST /company/employee
{
 "name": "Abdul4 Malik4",
 "age": 25,
 "experience": 3,
 "date": "2013-02-01",
 "FatherName" : "@Amar"
}

######## are same #######
GET /_search?q=_all:Amar
GET /_search?q=Amar

############################

POST /company/employee
{
 "name": "Abdul7 Malik7",
 "username": "abdul2013",
 "age": 25,
 "experience": 3,
 "date": "2014-02-01",
 "FatherName" : "@Amar"
}

POST /company/employee
{
 "name": "Abdul7 Malik7",
 "username": "abdul2013",
 "age": 25,
 "experience": 3,
 "date": "2014-02-01",
 "FatherName" : "@Amar"
}

POST /company/employee
{
 "name": "Abdul8 Malik8",
 "username": "abdul2013",
 "age": 25,
 "experience": 3,
 "date": "2013-02-01",
 "FatherName" : "@Amar"
}

POST /company/employee
{
 "name": "Abdul9 Malik9",
 "username": "abdul 2013",
 "age": 25,
 "experience": 3,
 "date": "2014-02-01",
 "FatherName" : "@Amar"
}

GET company/_mapping

GET company/_search?q=2013

GET company/_search?q=date:2013

POST /company/employee
{
 "name": "Andrew Thomas",
 "username": "Abdul9 Malik9",
 "age": 25,
 "experience": 3,
 "date": "2014-02-01",
 "FatherName" : "David Knol"
}

GET /_search?pretty
{
 "query": {
 "bool": {
 "must": {
 "match": {
 "name": "Andrew Gabbar"
 }
 },
 "should": {
 "range": {
 "age": {
 "gte": 15
 }
 }
 }
 }
 }
}

GET /_search?pretty=true
{
 
"query": { 
 "match": { 
 "name": { 
 "query": "Anant Sharma", 
 "operator": "or" 
 } 
 } 
}
}

POST _search
{
 "query": {
 "term" : { "username" : "abdul 2013" } 
 }
}

GET company/_search?pretty=true
{ 
 "query": { 
 "bool": { 
 "should": [ 
 { "term": { "username": "Amar" }}, 
 { "term": { "username": "abdul2013"}} 
 ] 
 }
 } 
}


PUT /myapp
{
 "mapping": {
 "tweet": {
 "properties": {
 "tweet": {
 "type": "string"
 },
 "name": {
 "type": "string"
 },
 "nick": {
 "type": "string"
 },
 "Date": {
 "type": "date", "format": "yyyy-MM-dd HH:mm:ss"
 },
 "rt": {
 "type": "long"
 }
 }
 }
 }
}

POST _analyze
{
 "analyzer": "whitespace",
 "text": "The quick brown fox."
}

POST _analyze
{
 "tokenizer": "standard",
 "filter": [ "lowercase", "asciifolding" ],
 "text": "Is this déja vu? A quick fox jump over lazy pumped"
}

POST _analyze
{
 "tokenizer": "standard",
 "filter": [ "lowercase", "asciifolding","stemmer" ],
 "text": "Is this déja vu? A quick fox jumped over lazy dog"
}

POST _analyze
{
 "tokenizer": "standard",
 "filter": [ "synonym", "asciifolding" ],
 "text": "Is this déja vu?"
}
POST _analyze
{
 "analyzer": "simple",
 "text": "The 5 quic4k brown fox."
}

POST _analyze
{
 "analyzer": "simple",
 "text": "Time2School."
}

Python Map Reduce Program

Mapper.py

#!/usr/bin/env python

import sys

for line in sys.stdin:

  line = line.strip()

  words = line.split()

for word in words:

  print '%s\t%s' % (word, 1)

Reducer.py

#!/usr/bin/env python

from operator import itemgetter

import sys

current_word = None

current_count = 0

word = None

# input comes from STDIN

for line in sys.stdin:

    line = line.strip()

    word, count = line.split('\t', 1)

    # convert count (currently a string) to int

    try:

        count = int(count)

    except ValueError:

        continue

    if current_word == word:

        current_count += count

    else:

        if current_word:

            print '%s\t%s' % (current_word, current_count)

        current_count = count

        current_word = word

if current_word == word:

    print '%s\t%s' % (current_word, current_count)

 

 

 

How to run –

  • Unix
    • echo “foo foo quux labs foo bar quux” | ~/Mapper.py
    • echo “foo foo quux labs foo bar quux” | ~/Mapper.py | sort -k1,1 | ~/Reducer.py
  • Hadoop
    • hadoop dfs -mkdir -p /tmp/synopsys/input
    • hadoop dfs -copuyFromLocal ~/example/WordCount1/file* /tmp/synopsys/input
    • hadoop dfs -ls  /tmp/synopsys/input/
    • /home/woir/hadoop-2.6.0/bin/hadoop jar /home/woir/hadoop-2.6.0/share/hadoop/tools/lib/hadoop-streaming-2.6.0.jar \
      -file /home/woir/Mapper.py -mapper /home/woir/Mapper.py \
      -file /home/woir/Reducer.py -reducer /home/woir/Reducer.py \
      -input /tmp/synopsys/input/* -output /user/amar/gutenberg-output1

 

 

 

Python Exercise 3

-------> file name hello.py
def foo():
    print "Hello Amar Sharma"

if __name__ == "__main__":
 foo()
 Exercise: Take the following list, nlis, and compute its average. That is, write a function 'average(numlis)' that uses a 'for' loop to sum up the numbers in numlis and divide by the length of numlis. Just to be sure that you got all the numbers in numlis, print each one in your 'for' loop and print the length of the the list. nlis = [2,4,8,105,210,-3,47,8,33,1] # average should by 41.5 Exercise : Write a function multiply(num1,num2) which prints the multiplication of the two numbers. Careful to take care of the input type.

------------------- Sample for Debugging and Profiler ---------------------
import re

import sys 

import time

def if_statement():

 """ Three slightly difference versions of if: if, if-else, if-elif-else"""

 time.sleep(1)

 x = 5

 y = 0

 z = 0

 if x > 0:

     print "x is positive" 

 if y > 0:

     print "y is positive" 

 else:

     print "y is not positive" 

 # elif can be repeated as often as necessary 

 if z > 0:

     print "z is positive" 

 elif z < 0:

     print "z is negative" 

 else:

     print "z must be 0" 

def foo():

    print "Hello Amar Sharma"

def hello():

 """ prints hello, world """

 print("Hello, world!")

if __name__ == "__main__":

 foo()

 hello()

 if_statement()




Sample Output 
  

Anants-MacBook-Pro:pythonexamplesset1 amars$ python -m pdb /tmp/d.py 
> /tmp/d.py(1)<module>()

-> import re

(Pdb) n

> /tmp/d.py(2)<module>()

-> import sys

(Pdb) n

> /tmp/d.py(3)<module>()

-> import time

(Pdb) n

> /tmp/d.py(5)<module>()

-> def if_statement():

(Pdb) n

> /tmp/d.py(28)<module>()

-> def foo():

(Pdb) n

> /tmp/d.py(30)<module>()

-> def hello():

(Pdb) n

> /tmp/d.py(34)<module>()

-> if __name__ == "__main__":

(Pdb) n

> /tmp/d.py(35)<module>()

-> foo()

(Pdb) s

--Call--

> /tmp/d.py(28)foo()

-> def foo():

(Pdb) s

> /tmp/d.py(29)foo()

-> print "Hello Amar Sharma"

(Pdb) s

Hello Amar Sharma

--Return--

> /tmp/d.py(29)foo()->None

-> print "Hello Amar Sharma"

(Pdb) s

> /tmp/d.py(36)<module>()

-> hello()

(Pdb) s

--Call--

> /tmp/d.py(30)hello()

-> def hello():

(Pdb) s

> /tmp/d.py(32)hello()

-> print("Hello, world!")

(Pdb) s

Hello, world!

--Return--

> /tmp/d.py(32)hello()->None

-> print("Hello, world!")

(Pdb) s

> /tmp/d.py(37)<module>()

-> if_statement()

(Pdb) s

--Call--

> /tmp/d.py(5)if_statement()

-> def if_statement():

(Pdb) s

> /tmp/d.py(8)if_statement()

-> time.sleep(1)

(Pdb) s

> /tmp/d.py(9)if_statement()

-> x = 5

(Pdb) p x

*** NameError: NameError("name 'x' is not defined",)

(Pdb) n

> /tmp/d.py(10)if_statement()

-> y = 0

(Pdb) p x

5

(Pdb) h

Documented commands (type help <topic>):

========================================

EOF    bt         cont      enable  jump  pp       run      unt   

a      c          continue  exit    l     q        s        until 

alias  cl         d         h       list  quit     step     up    

args   clear      debug     help    n     r        tbreak   w     

b      commands   disable   ignore  next  restart  u        whatis

break  condition  down      j       p     return   unalias  where 

Miscellaneous help topics:

==========================

exec  pdb

Undocumented commands:

======================

retval  rv

(Pdb) q
 

Python Exercise 2

------> file name hello.py
def foo():
    print "Hello Amar Sharma"

if __name__ == "__main__":
    foo()



Exercise-
Using List Comprehension get the list of vowels in word "INDIAN".

-------------------sample----------------------------
# Python Module example


def add(a, b):
   """This program adds two
   numbers and return the result"""
   result = a + b
   return result

Exercise - Make a module named mymath and add remaining function 
( multiply, division, subtraction in the module and call them 
in your program using following ways -
1. import mymath
2. from mymath import add,subtraction,division, multiplicaiton
3. from mymath import *


-------------------- sample class --------------------

class ComplexNumber:
 def __init__(self,r = 0,i = 0):
     self.real = r
     self.imag = i
 def getData(self):
     print("{0}+{1}j".format(self.real,self.imag))

# Create a new ComplexNumber object
c1 = ComplexNumber(2,3)

# Call getData() function
# Output: 2+3j
c1.getData()

Exercise - Create a class Employee with data members name, address and salary. 
Provide a method to display the salary and name of the employee. In the main 
program instantiate an object of the class and demonstrate the use of the class.


--------------------sample---------------------------------

Let's say you have a class ClassA which contains a method methodA defined as:

def methodA(self, arg1, arg2):
 # do something
and ObjectA is an instance of this class.
Now when ObjectA.methodA(arg1, arg2) is called, python internally converts it for you as:
ClassA.methodA(ObjectA, arg1, arg2)
The self variable refers to the object itself.
Exercise
Please read through a text file, split each line into its constituent
words, add each word to a dictionary, then add one to the number of times the
word occurs in the dictionary. Finally, we sort the dictionary and print it
out listing each word (key) and its count (value). 
import sys
filename = sys.argv[1]

def count_words(filename):
    """Reads through a text file and counts the number of appearances of each word."""
    # print("\n",filename,"\n") # You can check that the filename is correct
    text_file = open(filename) # open the file for reading
    # Set up an empty dictionary to start a standard design pattern loop
    words_dic = {}
    # This loop adds each word to the dictionary and updates its count. Change 
    # all words to lower case so Horse and horse are seen as the same word.
    for line in text_file: # step through each line in the text file
        for word in line.lower().split(): # split into a list of words
            word = word.strip("'?,.;!-/\"") # strip out the stuff we ignore
            if word not in words_dic:
                words_dic[word] = 0 # add word to words with 0 count
            words_dic[word] = words_dic[word] + 1 # add 1 to the count
    text_file.close()

    # Sorts the dictionary words into a list and then print them out
    print("List of words in the file with number of times each appears.")
    word_list = sorted(words_dic)
    for word in word_list:
         print(words_dic[word], word)


if __name__ == "__main__":
    count_words(filename)
#%%