Decrypt HTTPS traffic with Wireshark and Fiddler

×

Using Wireshark on Windows 7 – Key Facts and Overview

  • Wireshark is a traffic analyzer, that helps you to learn how networking work and how to diagnose Network problems
  • To track SSL Traffic on Windows we use Wireshark with Session Key Logging
  • If the Browser uses the Diffie-Hellman cipher we need to disable it

Understanding the SSL Handshake Protocol

TLS/SSL Handshake Diagram
Image https_img1.jpg NOT Found
  • The SSL or TLS client sends a “client hello” message that lists cryptographic information such as the SSL or TLS version and, in the client’s order of preference, the CipherSuites supported by the client
  • The SSL or TLS server responds with a “server hello” message that contains the CipherSuite chosen by the server from the list provided by the client
  • Reference:An overview of the SSL or TLS handshake
  • Note: The “server hello” is very important as it Diffie-Hellman Cipher can’t be traced !

Disable the Diffie-Hellman Cipher for Browsers

Disable the Diffie-Hellman Cipher for Firefox

  • Launch Firefox, navigate to “about:config” from address bar
  • Double click all the Diffie-Hellman ciphers starting with dhe or ecdhe to change the Value to false
  • Restart Firefox to take effect
Disable the Diffie-Hellman Cipher for Firefox
Image ssl_dh_img2.jpg NOT Found

Disable the Diffie-Hellman Cipher for Chrome

  • View the default Cipher Suites list from the Client Hello message of Chrome
  • Append all the ciphers that contain “ECDHE” or “DHE” in hex to command line parameter:”–cipher-suite-blacklist” and launch Chrome
  • Sample: chrome.exe –cipher-suite-blacklist=0xc02b,0xc02f,0x009e,0xcc14,0xcc13,0xc00a,0xc014,0x0039,0xc009,0xc013,0x0033
Cipher Suites (15 suites)
     Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
     Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
     Cipher Suite: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
     Cipher Suite: TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcc14)
     Cipher Suite: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcc13)
     Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
     Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
     Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
     Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
     Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
     Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
     Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
     Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
     Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
     Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)

Testing SSL Decryption with Wireshark

Setup Wireshark

Activate Session Key Logging on Windows Action
Image https_img1.jpg NOT Found
  • Create SSLKEYLOGFILE enviroment variable
Image https_img2.jpg NOT Found
  • Enable SSLKEYLOGFILE for SSL Protocol
Image https_img21.jpg NOT Found
  • If needed add add. HTTPS Ports like : Port 8443
  • Edit -> Preferences -> Protocols -> HTTP

Run Wireshark and decrypt a TLS/SSL packet

Run first https test against Google Website
Image https_img3.jpg NOT Found
  • Testing against https:/gooogle.de
  • The decrypted SSL indicates that this packet was decrypted sucessfully !

Advantages / Disadvantages using Wireshark

Advantages Disadvantages
A little bit complicated Very detailed Info for the complete Network Stack
You need to know a lot of Networking Details Client must support SSLKEYLOGFILE feature
Support only certain Browsers [ Firefox, Chrome ] and email clients [ thunderbird] No support for cURL and WordPress function like wp_get_remote() and

Reference

Using Fiddler

Fiddler – How it Works

Overview
Image fiddler_overview.jpg NOT Found
  • To read HTTPS data Fiddler is acting like a Man-in-the-Middle
  • Fiddler is listening to following Network proxy: localhost:8888
  • On startup Fiddler registers as a WinINET-Proxy service
  • All Browsers [ like Chrrome, Firefox ] taking care of this service sends its data to the >Fiddler-Proxy first
  • Finally Fiddler sends the data to the Target Host

Setup – Enable Decryption of SSL/HTTPS traffic

Setup
Image fiddler_setup.jpg NOT Found
  • Menu: Tools -> Option -> HTTPS
  • Check: X Decrypt HTTPS Traffic

Decrypt cURL HTTPS traffic with Fiddler

Using cUrl : add –proxy 127.0.0.1:8888 to the cUrl command
Image fiddler_curl1.jpg NOT Found
  • Start Fiddler
    [Note Fiddler uses Portnumber: 8888 per Default ]
  • Add –proxy 127.0.0.1:8888 to your curl command
Image fiddler_curl2.jpg NOT Found
  • Use Inspectors menu item
  • This gives as a decrypted view of the HTTPS Post Request
  • Note: Passwords and Username can now easily extracted
  • Use the RAW menu item to get details about
    HTTPS Response Request like:
    HTTP Response Code, HTTPS Header and HTTPS
    Response Body

Decrypt WordPress HTTPS traffic for wp_remote_get()/wp_remote_post() with Fiddler

Application Details

Display Timelines with Fiddler

Timeline
Image fiddler_timeline.jpg NOT Found
  • Page load [ /pv/wr] is decoupled from the PHP processing
  • PHP processing [initWRApp.php] is triggered via Ajax Request
  • All Remote PHP processing steps [login.action, tologin.action and summaryInfo.action] runs sequentially

Display HTTP Packet Details with Fiddler

Step1 : Initial Page Load WebBrowser -> Local Webserver [ 192.168.1.50]
Image fiddler_data_img1.jpg NOT Found
  • Protocal: HTTP
  • Http Get Request to load the initial Page
Step2: Starting PHP Remote Processing WebBrowser -> Local Webserver [ 192.168.1.50]
Image fiddler_data_img2.jpg NOT Found
  • Protocal: HTTP
  • Http Get Request to start PHP Remote Processing with wp_remote_get and wp_remote_post
Step3: Authenticate with Username/Password against Remote Webserver Local Webserver [ 192.168.1.50] -> Remote Webserver [ 52.58.164.53:8443 ]
Image /fiddler_data_img3.jpg NOT Found
  • Protocol: HTTPS Post
  • URL: https://52.58.164.53:8443/security!login.action
  • WordPress Function used: wp_remote_post()
  • Remote Webserver return “op.successfully” if login OK !
  • Remote Webserver return New JSESSIONID cookie if login OK !
Step 4: Capture initial Page after Login into Remote Werbserver Local Webserver [ 192.168.1.50] -> Remote Webserver [ 52.58.164.53:8443]
Image fiddler_data_img4.jpg NOT Found
  • Protocol: HTTPS Get
  • URL: https://52.58.164.53:8443/security!tologin.action
  • WordPress Function used: wp_remote_get()
  • Remote Webserver return Initial Webpage after successful login!
  • Remote Webserver return X-CSRF-Token which should be used for subsequent requests
Step 5: Finally capture Data from Remote Werbserver Local Webserver [ 192.168.1.50] -> Remote Webserver [ 52.58.164.53:8443]
Image fiddler_data_img5.jpg NOT Found
  • Protocol: HTTPS Post
  • URL: https://52.58.164.53:8443/summaryAction!querySummaryInfo.action?nodeSN=0
  • WordPress Function used: wp_remote_post()
  • Remote Webserver return Data in Json Format
  • For this request we use JSESSIONID cookie returned from Step 3
  • For this request we use X-CSRF-TOKEN returned from Step 4

Advantages/Disadvantages using Fiddler

Advantages Disadvantages
Easy Setup Only HTTP/HTTP traffic can be monitored
Support all sort of clients like cURl, PHP, Browers, Email Clients, …

Automate Login to a CRSF protected WebSite using cURL

×

Tools Used for this Tutorial

Tool Tool Version
Curl 7.56.1 (x86_64-pc-win32) libcurl/7.56.1 OpenSSL/1.1.0g (WinSSL)
Google Chrome Version 61.0.3163.100 (Official Build)
Google Dev Tools Version 61.0.3163.100 (Official Build)
bash shell 4.4.12(3)-release

Overview – Login to a WebSite in 3 Steps

Script Name Action Return
WRlogin.sh Login to WebSite using Username Password JSESSIONID cookie
WRlogin2.sh Redirect Request after sucessfull Login X-CSRF-TOKEN via HTML Meta Tags
WRData.sh Access the login protected WebSite Inverter Data like: Current Power, ..

Curl Usage with Google Dev Tools – First steps

To copy Curl Syntax for bash Shell do the following

  • Load your inital WebSite a first time
  • Press F12 to attach/open Google Dev Tools
  • Reload your Page
  • Navigate to Network Tab
  • Right click the desired API call
  • Select “Copy” -> “Copy as cURL (bash)”
Retrieve the cURL bash Command using Google Dev Tools
Image curl_img1.jpg NOT Found

Run our first cURL command

  • Paste the cURL command to a bash Shell
  • Add -v switch to get HTTP header dumped
  • redirect stderr by adding: 2>&1
$ curl 'https://52.58.164.53:8443/index.action'  -H 'Pragma: no-cache' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.8' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Cache-Control: no-cache' -H 'Cookie: JSESSIONID=1jzw19tuvg42wl93fxp1exzkf'  -H 'Connection: keep-alive' --compressed --insecure -v 2>&1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
...
{ [5 bytes data]
< HTTP/1.1 200 OK
< Expires: Thu, 01-Jan-1970 00:00:00 GMT
< Set-Cookie: JSESSIONID=ioyf0enlkduy1geo82r2tryk8;Path=/;Secure;HttpOnly
< Cache-Control: no-store,no-cache
< X-Frame-Options: SAMEORIGIN
< X-Download-Options: noopen
< X-XSS-Protection: 1;mode=block
< Strict-Transport-Security: max-age=31536000;includeSubdomains
< X-Content-Type-Options: nosniff
< Content-Language: de-DE
< Content-Type: text/html; charset=utf-8
< Content-Length: 8471
...

HTTP Status of our first cURL request

  • The Website gets loaded sucessfuly -> HTTP/1.1 200 OK
  • The Website Content-Length is: 8471
  • The Website uses cookies: JSESSIONID
  • The Website uses https protocol

Use the Google Dev to understand the Login Logic

JavaScript and JavaScript Files load by the Initial Request
Image curl_img2.jpg NOT Found

index.action loads 2 javascript files

  • login.js
  • verifyCode.js

Processing and Validation of Login Form


	var verifyResult;
	$("#login").click(function() {
		if (!allowLogin) {
			jAlert(browserMessage, Message.alarm_info, '{"' + Message.sur						+ '":"OK"}');
			clearPassword();				
			return;
		}

		var userName = $.trim($("#userName").val());
		var password = $("#password").val();

		if (!loginValidate(userName, password)) {
			clearPassword();
			return;
		}
		var showVerifyCode = document.getElementById("verifyCode_tr").style.display;
		comitLogin(userName, password);
	});
  • For our cURL tutorials we can skip this Step as we already know username / password
  • Let’s review the JS function comitLogin()

Login Code implemented via sync. AJAX POST resquest

function comitLogin(userName, password) {

	if ($("#veryCode").is(":visible")) {
		if (!validateVCodeLength()) {
			clearPassword();
			return;
		}
	}

	var code = $("#veryCode").attr("value");

	$.ajax({
		url : basePath + "security!login.action",
		type : "POST",
		async : false,
		dataType : "json",
		data : {
			"userName" : userName,
			"password" : password,
			dateTime : new Date().getTime(),
			"veryCode" : code
		},
		cache : false,
		beforeSend:function (XMLHttpRequest)
		{
			//let go
		},
		success : function(res) {
			clearPassword();
			$("#pass1").val("");
			$("#pass2").val("");
			$("#loginErrorMsg").html("")
			$("#loginErrorMsg2").html("");
			if (res.retMsg == "op.successfully") {
				window.location.href = newPath + "securitys!tologin.action";
				return;
			} else if (res.retMsg == "op.update") {
				  $("#loginView").hide();
                       ....
  • For sending username,password,date and verificationCode a POST request is used
  • The remote function to be called is security!login.action
  • A successfull login should return: “op.successfully”
  • After successfull login a page redirection is triggered to: securitys!tologin.action

Step 1: Implement the Website JavaScript Login with bash/cURL

Set breakpoint in AJAX POTS when returning from AJAX POST request
Image curl_img3.jpg NOT Found
Login the Page – Login process is paused
Image curl_img4.jpg NOT Found
Verify Response and Request Headers
Image curl_img5.jpg NOT Found
  • For further processing we need to use the JSESSIONID returned by the Response header

Paste above Output to a Bash Shell

$  curl 'https://52.58.164.53:8443/security!login.action' -H 'Pragma: no-cache' -H 'Origin: https://52.58.164.53:8443' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Cache-Control: no-cache' -H 'X-Requested-With: XMLHttpRequest' -H 'Cookie: JSESSIONID=jhqdzlufdh95hevd878z5nig' -H 'Connection: keep-alive' -H 'Referer: https://52.58.164.53:8443/index.action' --data 'userName=pvlocal&password=PPPP&dateTime=1510764929706&veryCode=' --compressed
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

 -> Disable SSL verification by using -k switch 

$ curl 'https://52.58.164.53:8443/security!login.action' -H 'Pragma: no-cache' -H 'Origin: https://52.58.164.53:8443' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Cache-Control: no-cache' -H 'X-Requested-With: XMLHttpRequest' -H 'Cookie: JSESSIONID=jhqdzlufdh95hevd878z5nig' -H 'Connection: keep-alive' -H 'Referer: https://52.58.164.53:8443/index.action' --data 'userName=pvlocal&password=PPPP&dateTime=1510764929706&veryCode=' --compressed -k -i
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    95  100    28  100    67     28     67  0:00:01 --:--:--  0:00:01   196HTTP/1.1 200 OK
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Set-Cookie: JSESSIONID=1pt4jamjah1vbl44uep5ecvzm;Path=/;Secure;HttpOnly
Cache-Control: no-store,no-cache
X-Frame-Options: SAMEORIGIN
X-Download-Options: noopen
X-XSS-Protection: 1;mode=block
Strict-Transport-Security: max-age=31536000;includeSubdomains
X-Content-Type-Options: nosniff
Content-Language: de-DE
Set-Cookie: JSESSIONID=a5sjmaztpwa81acg06hymnxgb;Path=/;Secure;HttpOnly
Content-Type: application/json;charset=UTF-8
Content-Length: 28

{"retMsg":"op.successfully"}

 -> cURL Request works Now  

Testing an Invalid Login by changing password to xxxx

$ curl 'https://52.58.164.53:8443/security!login.action' -H 'Pragma: no-cache' -H 'Origin: https://52.58.164.53:8443' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Cache-Control: no-cache' -H 'X-Requested-With: XMLHttpRequest' -H 'Cookie: JSESSIONID=jhqdzlufdh95hevd878z5nig' -H 'Connection: keep-alive' -H 'Referer: https://52.58.164.53:8443/index.action' --data 'userName=pvlocal&password=xxxx&dateTime=1510764929706&veryCode=' --compressed -k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   125  100    58  100    67     58     67  0:00:01 --:--:--  0:00:01   348 {"retMsg":"Falscher Benutzername oder falsches Passwort."}

Create a Bash Script for Automation – WRlogin.sh

#!/bin/bash
#
# Use Google Dev dev tools and select Copy as cURL (bash) to learn quickly about the Curl Syntax for a HTTP request
#
# After a successfull login a NEW  JSESSIONID returned from the  response cookie should be used for subsequent requests
#

loginCredentials="userName=pvlocal&password=Feli2010&dateTime=1510483708371&veryCode"
url="https://52.58.164.53:8443/security!login.action"

echo "--------------------------------------------------------------------------------------------------------"
echo "-> URL                      :" $url
echo "-> INPUT Cookie             :" $cookie
echo "-> INPUT Login Credentials  :" $loginCredentials
echo "-> OUTPUT                   :" NEW JSESSIONID cookie after successfull Login
echo "--------------------------------------------------------------------------------------------------------"
#
# DebugON  set, possibly to the empty string will printout add. trace info
# To enable Debugging run:  $ export DebugON
#
if [ $DebugON ]; then
    echo "DebugON is set, possibly to the empty string"
    set -x
fi
#

# Only use the needed HTTP header Fields - Most Http Header Fields copied from our Google DEV Tool cURL copy we do not need
curl_header1='Accept-Language: en-US,en;q=0.8'
curl_header2='Accept-Encoding: gzip, deflate, br'
curl_header3='Accept: application/json, text/javascript, */*; q=0.01'
#
#  Arrays makes this much easier. Don't use Quotes here as $cookie and $csrf_token content uses spaces !
#
args=("-k" "-v" "$url" -H "$cookie" "--data" "$loginCredentials"  -H "$csrf_token"  -H "$curl_header1" -H "${curl_header2}"  -H "${curl_header3}")
#
echo "---------------------------------- cURL command to be executed -----------------------------------------"
echo curl   "${args[@]}"
echo "--------------------------------------------------------------------------------------------------------"
output=$( curl "${args[@]}" 2>&1 )

#
# Sucessfull Login returns following string
#    '"retMsg":"op.successfully"}
#

login_status=""
login_status=$(echo "$output" | grep  retMsg | awk -v FS="(retMsg\":\"|\"})"  '{print  $2 }')

#
# Note; login.sh return two occurances of string:
#   Set-Cookie: JSESSIONID=h0eaof37hnta1wtjlk1j1nig2;Path=/;Secure;HttpOnly
# As we are only intrested on the 2.nd one use: sed -n 2p
#

expcmd=""
expcmd=$(echo "$output" | grep Set-Cookie | sed -n 2p | awk -v FS="(Set-Cookie: |;)"  '{print "export cookie=\"Cookie: " $2 "\""}')

http_return=""
http_return=$(echo "$output" |  grep '< HTTP')


#echo "Login Status                 : " "$login_status"
# echo "New JSESSIONID EXPORT command: "  "$expcmd"

if [ "$login_status" == "op.successfully" ]; then
        echo "--------------------------------------------------------------------------------------------------------"
        echo "HTTP Return Code: "  "$http_return"
        echo "Login OK : Javascript Return Status:" "$login_status"
        echo "Run New JSESSIONID EXPORT command:  \$" "$expcmd"
        echo "--------------------------------------------------------------------------------------------------------"
elif [ "$login_status" == "op.verifyCode.fail" ]; then
        echo "--------------------------------------------------------------------------------------------------------"
        echo "HTTP Return Code: "  "$http_return"
        echo "Login Failded - Wrong verification Code  : Javascript Return Status:" "$login_status"
        echo "Login to WebSite and enter verification Code - after successfull login rerun this script !"
        echo "--------------------------------------------------------------------------------------------------------"
else
        echo "--------------------------------------------------------------------------------------------------------"
        echo "HTTP Return Code: "  "$http_return"
        echo "Login failed  check Login credentials - Javascript Return Status: " "$login_status"
        echo "--------------------------------------------------------------------------------------------------------"
fi

Testing the bash Script WRlogin.sh

$ ./WRlogin.sh
--------------------------------------------------------------------------------------------------------
-> URL                      : https://52.58.164.53:8443/security!login.action
-> INPUT Cookie             : Cookie: JSESSIONID=17grbvbtwu2ga9blndj7rfrsk
-> INPUT Login Credentials  : userName=pvlocal&password=Feli2010&dateTime=1510483708371&veryCode
-> OUTPUT                   : NEW JSESSIONID cookie after successfull Login
--------------------------------------------------------------------------------------------------------
---------------------------------- cURL command to be executed -----------------------------------------
curl -k -v https://52.58.164.53:8443/security!login.action -H Cookie: JSESSIONID=17grbvbtwu2ga9blndj7rfrsk --data userName=pvlocal&password=Feli2010&dateTime=1510483708371&veryCode -H X-CSRF-TOKEN: c3d70f5d-4388-4d85-a1db-9ac797b92418 -H Accept-Language: en-US,en;q=0.8 -H Accept-Encoding: gzip, deflate, br -H Accept: application/json, text/javascript, */*; q=0.01
--------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------
HTTP Return Code:  < HTTP/1.1 200 OK
Login OK : Javascript Return Status: op.successfully
Run New JSESSIONID EXPORT command:  $ export cookie="Cookie: JSESSIONID=rudqwfhaqx0m1mchkau9ycqzs"
--------------------------------------------------------------------------------------------------------

What we learned in Step 1

  • A sucessful login returns message {"retMsg":"op.successfully"} and a new JSESSIONID
  • This JSESSIONID need to be used for subsequent HTTP/AJAX request
  • Note: At this stage we are able to attack this page using a brute FORCE methode

Step 2: Implement the Login Redirect using bash/cURL

  • After as succesfull login the page is redirectet to securitys!tologin.action
  • For the complete JS Code review Step 1
    if (res.retMsg == "op.successfully") {
	window.location.href = newPath + "securitys!tologin.action";
    return;

Use Google Dev tools to extract cURL bash Code

  • Login to the WebSite
  • Navigate to Network Tab
  • Locate the securitys!tologin.action Request
  • Select “Copy” -> “Copy as cURL (bash)”
Initial page
Image curl_img6.jpg NOT Found

Paste the Code to a bash Shell

$  curl 'https://52.58.164.53:8443/securitys!tologin.action' -H 'Pragma: no-cache' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Referer: https://52.58.164.53:8443/index.action' -H 'Cookie: JSESSIONID=1b0miyons0sfl1uqxsbhdv0zud' -H 'Connection: keep-alive' -H 'Cache-Control: no-cache' --compressed --insecure
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

<br /> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><br /> <html><br /> <head><br /> <base href="https://52.58.164.53:8443/"/><br /> <title>NetEco 1000S</title><br /> <meta http-equiv="pragma" content="no-cache"/><br /> <meta http-equiv="cache-control" content="no-cache"/><br /> <meta http-equiv="expires" content="0"/><br /> <meta http-equiv="keywords" content="keyword1,keyword2,keyword3"/><br /> <meta http-equiv="description" content="This is my page"/><br /> <meta http-equiv="X-UA-Compatible" content="IE=11,IE=8" /><br /> <meta name="_csrf" content="508a5c53-6d3d-4ed1-b56e-e292fcf0a2dc"/><br /> <meta name="_csrf_header" content="X-CSRF-TOKEN"/><br />

Keep special attention to _crsf HTML meta tags returned from above cURL request

  • Note this CRSF Tojne will be used to access the Website !
 meta name="_csrf" content="508a5c53-6d3d-4ed1-b56e-e292fcf0a2dc" 
 meta name="_csrf_header" content="X-CSRF-TOKEN" 

Build script WRLogin2.sh to simulate the Page redirect


#!/bin/bash
#
# Use Googel Dev dev tools and select Copy as cURL (bash) to learn quickly about the Curl Syntax for a HTML request
#
# Use curl -i to get the response header dumped for this GET request
#       This allows us to track the response cookies
# Usage :
#       This script uses the new  JSESSIONID  returned from login.sh
#       This script returns the csrf_token which can be finally used for all subseqent server connections
#           export csrf_token="X-CSRF-TOKEN: 59bbc7b3-5c50-4220-b609-9506aaa83ea4"
#

url="https://52.58.164.53:8443/securitys!tologin.action"
echo "--------------------------------------------------------------------------------------------------------"
echo "-> URL                      :" $url
echo "-> INPUT Cookie JSESSIONID  :" $cookie
echo "-> OUTPUT                   :" X-CSRF-TOKEN extract from HTML META _csrf
echo "--------------------------------------------------------------------------------------------------------"
# DebugON  set, possibly to the empty string will printout add. trace info
# To enable Debugging run:  $ export DebugON
#
if [ $DebugON ]; then
    echo "DebugON is set, possibly to the empty string"
    set -x
fi
#
# Only use the needed HTTP header Fields - Most Http Header Fields copied from our Google DEV Tool cURL copy we do not need
curl_header1='Accept-Language: en-US,en;q=0.8'
curl_header2='Accept-Encoding: gzip, deflate, br'
curl_header3='Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8'

#
#  Arrays makes this much easier. Don't use Quotes here as $cookie and $csrf_token content uses spaces !
#
args=("-k" "-v" "$url" -H "$cookie"  -H "$csrf_token"  -H "$curl_header1" -H "${curl_header2}"  -H "${curl_header3}")
#
echo "---------------------------------- cURL command to be executed -----------------------------------------"
echo "curl "  "${args[@]}"
echo "--------------------------------------------------------------------------------------------------------"
#
# Execute Curl Command and save stdout and stderr to a Shell variable
output=$( curl "${args[@]}" 2>&1 )
#
# login2.sh retuns only a NEW X-CSRF-TOKEN via response META Tags _crsf
# We need to extract the crsf_token from the HTML repsonse Meta header  and use if for furhter requests together with JSESSIONID !
#    
#        

#
# - Rerunning login2.sh may result that _crsf meta will be returne 2x.
#    ->Just stop the awk script if we have found X-CSRF-TOKEN
#   by callung exit.
# - Note we search for string _csrf" to limit the lines returned
#

csrf_token=""
csrf_token=$(echo "$output" |  grep '_csrf\" content'  | awk -v FS="(content=\"|\"/>)"  '{print "export csrf_token=\"X-CSRF-TOKEN: "  $2 "\""; exit }' )

http_return=""
http_return=$(echo "$output" |  grep '< HTTP')

if [ -n "$csrf_token" ]; then
        echo "--------------------------------------------------------------------------------------------------------"
        echo "HTTP Return Code: " $http_return
        echo "            Note:  Dont forget to set \$csrf_token variable extracted from HTML Meta Data  !  "
        echo "             Run:  \$" "$csrf_token"
        echo "--------------------------------------------------------------------------------------------------------"
else
        echo "--------------------------------------------------------------------------------------------------------"
        echo "ERROR : X-CSRF-TOKEN not found !"
        echo "HTTP Return Code: " $http_return
        echo "--------------------------------------------------------------------------------------------------------"

fi

Test script WRLogin2.sh to simulate the Page redirect

  • WRlogin2.sh extracts the X-CSRF-TOKEN from HTML Meta Tags
  • Together with JSESSIONID return from WRLogin.sh this Info is used to access our target WEBSite
$ ./WRlogin2.sh
--------------------------------------------------------------------------------------------------------
-> URL                      : https://52.58.164.53:8443/securitys!tologin.action
-> INPUT Cookie JSESSIONID  : Cookie: JSESSIONID=rudqwfhaqx0m1mchkau9ycqzs
-> OUTPUT                   : X-CSRF-TOKEN extract from HTML META _csrf
--------------------------------------------------------------------------------------------------------
---------------------------------- cURL command to be executed -----------------------------------------
curl  -k -v https://52.58.164.53:8443/securitys!tologin.action -H Cookie: JSESSIONID=rudqwfhaqx0m1mchkau9ycqzs -H X-CSRF-TOKEN: c3d70f5d-4388-4d85-a1db-9ac797b92418 -H Accept-Language: en-US,en;q=0.8 -H Accept-Encoding: gzip, deflate, br -H Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
--------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------
HTTP Return Code:  < HTTP/1.1 200 OK
            Note:  Dont forget to set $csrf_token variable extracted from HTML Meta Data  !
             Run:  $ export csrf_token="X-CSRF-TOKEN: 218df123-fa18-44ba-8f3f-2dbc378e987c"
--------------------------------------------------------------------------------------------------------

Step 3: Finally Implement the Page Access using bash/cURL

  • For this script we need: JSESSIONID returned from WRlogin.sh
  • X-CSRF-TOKEN retuned from WRlogin2,sh

Follow the steps from above to extract the cURL bash command

$ curl 'https://52.58.164.53:8443/summaryAction!querySummaryInfo.action?nodeSN=0' -X POST -H 'Pragma: no-cache' -H 'Origin: https://52.58.164.53:8443' -H 'Accept-Encoding: gzip, deflate, br' -H 'X-CSRF-TOKEN: 508a5c53-6d3d-4ed1-b56e-e292fcf0a2dc' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Cache-Control: no-cache' -H 'X-Requested-With: XMLHttpRequest' -H 'Cookie: JSESSIONID=1b0miyons0sfl1uqxsbhdv0zud' -H 'Connection: keep-alive' -H 'Referer: https://52.58.164.53:8443/summaryAction!accessSummaryPage.action?nodeSN=0' -H 'Content-Length: 0' --compressed --insecure
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1932  100  1932    0     0   1932      0  0:00:01 --:--:--  0:00:01  7263

...... -->Respone is JSON Encoded 

"summaryInfo":"{\"batterySurfaceTemp\":\"\",\"co2reduce\":\"8,062 t\",\"converseNum\":\ ":\"EUR\",
\"currentPower\":\"3,860 kW\",\"dayPower\":\"12,170 kWh\",\"daytotalRadiation\":\"\",\"gridStatus\":false
.....

Build a bash shell script WRData.sh for simulating the WebSite Access

#!/bin/bash
#
# Use Google Dev dev tools and select Copy as cURL (bash) to learn quickly about the Curl Syntax for a HTML request
#
# Note to run this script we need a valid Env - The related export commands should be run after script executipn
#
#    - JSESSIONID cookie -> login.sh  -> export cookie="Cookie: JSESSIONID=18knsdija6tyjjmej043tac8m"
#    - X-CSRF-TOKEN      -> login2.sh -> export csrf_token='X-CSRF-TOKEN: 1de4c25b-724a-4b96-a187-c72b134a5b9f'
#
#  --> When now rerunning this SCRIPT both CSRF data and JSESSIONID should not change anymore !

url="https://52.58.164.53:8443/summaryAction%21querySummaryInfo.action?nodeSN=0"
echo "--------------------------------------------------------------------------------------------------------"
echo "-> URL                      :" $url
echo "-> INPUT Cookie JSESSIONID  :" $cookie
echo "-> INPUT X-CSRF-TOKEN       :" $csrf_token
echo "-> OUTPUT                   :" HTML Content
echo "--------------------------------------------------------------------------------------------------------"

# DebugON  set, possibly to the empty string will printout add. trace info
if [ $DebugON ]; then
    echo "DebugON is set, possibly to the empty string"
    set -x
fi
#
# Only use the needed HTTP header Fields - Most Http Header Fields copied from our Google DEV Tool cURL copy we do not need
curl_header1='Accept-Language: en-US,en;q=0.8'
curl_header2='Accept-Encoding: gzip, deflate, br'
curl_header3='Accept: application/json, text/javascript, */*; q=0.01'
#
#  Arrays makes this much easier. Don't use Quotes here as $cookie and $csrf_token content uses spaces !
#
args=("-k" "-v" "-X" "POST" "$url" -H "$cookie"  -H "$csrf_token"  -H "$curl_header1" -H "${curl_header2}"  -H "${curl_header3}")
#
echo "---------------------------------- cURL command to be executed -----------------------------------------"
echo "curl "  "${args[@]}"
echo "--------------------------------------------------------------------------------------------------------"
output=$( curl "${args[@]}" 2>&1 )
#
#set +x
#
# String operation on Output
#    awk -v FS="(currentPower|dayPower)"  '{print $2 }'                            ->  \":\"20,397 kW\",\"
#    sed 's/...$//' --> removes the 3 last character [ including the 2.nd Commma ] ->  \":\"20,397 kW\"
#    | tr -d '\\":' --> removes \": from the remaing string :                      ->  20,397 kW
#
# Note : Don't use kW or kWh as FS for the awk command as this may break awk due to multiple occurance of these strings
#        in the HTMT reponse !
#
current_power=""
current_power=$(echo "$output" |  grep 'summaryInfo'  | awk -v FS="(currentPower|dayPower)"  '{print $2 }'  | sed 's/...$//' | tr -d '\\":' )

day_power=""
day_power=$(echo "$output" |  grep 'summaryInfo'  | awk -v FS="(dayPower|daytotalRadiation)"  '{print $2 }' |  sed 's/...$//' | tr -d '\\":' )

http_return=""
http_return=$(echo "$output" |  grep '< HTTP')
echo "--------------------------------------------------------------------------------------------------------"
date
echo "HTTP Return Code: " $http_return
echo "Current Power   : " $current_power
echo "    Day Power   : " $day_power
echo "--------------------------------------------------------------------------------------------------------"

Test bash shell script WRData.sh to access our target WebSite

$ ./WRData.sh
--------------------------------------------------------------------------------------------------------
-> URL                      : https://52.58.164.53:8443/summaryAction%21querySummaryInfo.action?nodeSN=0
-> INPUT Cookie JSESSIONID  : Cookie: JSESSIONID=rudqwfhaqx0m1mchkau9ycqzs
-> INPUT X-CSRF-TOKEN       : X-CSRF-TOKEN: 218df123-fa18-44ba-8f3f-2dbc378e987c
-> OUTPUT                   : HTML Content
--------------------------------------------------------------------------------------------------------
---------------------------------- cURL command to be executed -----------------------------------------
curl  -k -v -X POST https://52.58.164.53:8443/summaryAction%21querySummaryInfo.action?nodeSN=0 -H Cookie: JSESSIONID=rudqwfhaqx0m1mchkau9ycqzs -H X-CSRF-TOKEN: 218df123-fa18-44ba-8f3f-2dbc378e987c -H Accept-Language: en-US,en;q=0.8 -H Accept-Encoding: gzip, deflate, br -H Accept: application/json, text/javascript, */*; q=0.01
--------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------
Fr, 17. Nov 2017 10:09:24
HTTP Return Code:  < HTTP/1.1 200 OK
Current Power   :  3,846 kW
    Day Power   :  3,770 kWh
--------------------------------------------------------------------------------------------------------

Final test of our developed Shell Scripts

$ ./WRlogin.sh
--------------------------------------------------------------------------------------------------------
-> URL                      : https://52.58.164.53:8443/security!login.action
-> INPUT Cookie             : Cookie: JSESSIONID=rudqwfhaqx0m1mchkau9ycqzs
-> INPUT Login Credentials  : userName=pvlocal&password=Feli2010&dateTime=1510483708371&veryCode
-> OUTPUT                   : NEW JSESSIONID cookie after successfull Login
--------------------------------------------------------------------------------------------------------
---------------------------------- cURL command to be executed -----------------------------------------
curl -k -v https://52.58.164.53:8443/security!login.action -H Cookie: JSESSIONID=rudqwfhaqx0m1mchkau9ycqzs --data userName=pvlocal&password=Feli2010&dateTime=1510483708371&veryCode -H  -H Accept-Language: en-US,en;q=0.8 -H Accept-Encoding: gzip, deflate, br -H Accept: application/json, text/javascript, */*; q=0.01
--------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------
HTTP Return Code:  < HTTP/1.1 200 OK
Login OK : Javascript Return Status:  op.successfully 
Run New JSESSIONID EXPORT command:  $ export cookie="Cookie: JSESSIONID=slz60pg4qbqp1bxj6gcrfhgdw" 
--------------------------------------------------------------------------------------------------------

$  export cookie="Cookie: JSESSIONID=slz60pg4qbqp1bxj6gcrfhgdw"
$ ./WRlogin2.sh
--------------------------------------------------------------------------------------------------------
-> URL                      : https://52.58.164.53:8443/securitys!tologin.action
-> INPUT Cookie JSESSIONID  : Cookie: JSESSIONID=slz60pg4qbqp1bxj6gcrfhgdw
-> OUTPUT                   : X-CSRF-TOKEN extract from HTML META _csrf
--------------------------------------------------------------------------------------------------------
---------------------------------- cURL command to be executed -----------------------------------------
curl  -k -v https://52.58.164.53:8443/securitys!tologin.action -H Cookie: JSESSIONID=slz60pg4qbqp1bxj6gcrfhgdw -H  -H Accept-Language: en-US,en;q=0.8 -H Accept-Encoding: gzip, deflate, br -H Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
--------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------
HTTP Return Code:  < HTTP/1.1 200 OK
            Note:  Dont forget to set $csrf_token variable extracted from HTML Meta Data  !
             Run:  $  export csrf_token="X-CSRF-TOKEN: 3fa4555c-0890-4806-a7f1-ad0ffcf6fa41" 
--------------------------------------------------------------------------------------------------------

$ export csrf_token="X-CSRF-TOKEN: 3fa4555c-0890-4806-a7f1-ad0ffcf6fa41"
$ ./WRData.sh
--------------------------------------------------------------------------------------------------------
-> URL                      : https://52.58.164.53:8443/summaryAction%21querySummaryInfo.action?nodeSN=0
-> INPUT Cookie JSESSIONID  : Cookie: JSESSIONID=slz60pg4qbqp1bxj6gcrfhgdw
-> INPUT X-CSRF-TOKEN       : X-CSRF-TOKEN: 3fa4555c-0890-4806-a7f1-ad0ffcf6fa41
-> OUTPUT                   : HTML Content
--------------------------------------------------------------------------------------------------------
---------------------------------- cURL command to be executed -----------------------------------------
curl  -k -v -X POST https://52.58.164.53:8443/summaryAction%21querySummaryInfo.action?nodeSN=0 -H Cookie: JSESSIONID=slz60pg4qbqp1bxj6gcrfhgdw -H X-CSRF-TOKEN: 3fa4555c-0890-4806-a7f1-ad0ffcf6fa41 -H Accept-Language: en-US,en;q=0.8 -H Accept-Encoding: gzip, deflate, br -H Accept: application/json, text/javascript, */*; q=0.01
--------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------
Fr, 17. Nov 2017 12:24:10
HTTP Return Code:  < HTTP/1.1 200 OK
Current Power   :  3,949 kW
    Day Power   :  13,890 kWh
--------------------------------------------------------------------------------------------------------

$ ./WRData.sh
--------------------------------------------------------------------------------------------------------
-> URL                      : https://52.58.164.53:8443/summaryAction%21querySummaryInfo.action?nodeSN=0
-> INPUT Cookie JSESSIONID  : Cookie: JSESSIONID=slz60pg4qbqp1bxj6gcrfhgdw
-> INPUT X-CSRF-TOKEN       : X-CSRF-TOKEN: 3fa4555c-0890-4806-a7f1-ad0ffcf6fa41
-> OUTPUT                   : HTML Content
--------------------------------------------------------------------------------------------------------
---------------------------------- cURL command to be executed -----------------------------------------
curl  -k -v -X POST https://52.58.164.53:8443/summaryAction%21querySummaryInfo.action?nodeSN=0 -H Cookie: JSESSIONID=slz60pg4qbqp1bxj6gcrfhgdw -H X-CSRF-TOKEN: 3fa4555c-0890-4806-a7f1-ad0ffcf6fa41 -H Accept-Language: en-US,en;q=0.8 -H Accept-Encoding: gzip, deflate, br -H Accept: application/json, text/javascript, */*; q=0.01
--------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------
Fr, 17. Nov 2017 12:24:16
HTTP Return Code:  < HTTP/1.1 200 OK
Current Power   :  3,949 kW
    Day Power   :  13,890 kWh
--------------------------------------------------------------------------------------------------------

Reference

  • DVWA - Main Login Page - Brute Force HTTP POST Form With CSRF Token
  • Using GIT

    ×

    Tutorial and Download Location

    GIT Tutorial https://www.youtube.com/watch?v=SWYqp7iY_Tc&t=8s
    Download Location for Windows https://git-scm.com/download/win

    GIT Concepts & Commands

    Concepts First Commands
    Image git_img1.jpg NOT Found Image git_img2.jpg NOT Found

    Using GIT bash and print version and help panel

    GIT Bash Window
    Image git_img3.jpg NOT Found
  • First Git Commands
  • GIT Functions

    Working Directory, Staging Area and Repositories
    Image git_overview.jpg NOT Found
  • Above image is copied from: https://sc5.io/posts/how-to-manage-your-wordpress-codebase-with-git/#gref
  • Init Local Git Repository

    Setup Working Directory, Staging Area and Local Repository
    Image git_img4.jpg NOT Found Run Scripts:

  • /d/GIT/gitInit.sh – Init/Reset GIT repository


  • Init Git Repository

  • The Local Git Repository is stored in .git directory
  • To remove the Git Repository run: rm -rf .git
  • Using .gitignore

    Specify intentionally untracked files to be ignored
    Image git_img5.jpg NOT Found Run Scripts:

  • /d/GIT/gitIngnore.sh
  • Mark files/directories to be igrore by GIT operations

  • Adding some files to the Local Reposiory by running git commit

    Adding 2 files to the working area Details
    Image chapt1_img1.jpgNOT Found Run Scripts:

  • /d/GIT/gitCommit.sh – Add diffs to our Local GIT repository


  • Step 1

  • Adding 2 files to the staging area
  • Commit the changes Details
    Image chapt1_img2.jpgNOT Found Step 2: Commit the data

  • After git commit the staging area gets cleaned
  • Create an GitHup account and setup an GIT Remote Repository

    Create an empty GitHub Remote Repository
    Image git_chapt2_img1.jpg NOT Found
    Image git_chapt2_img2.jpg NOT Found
    Image git_chapt2_img3.jpg NOT Found Run Scripts:

  • /d/GIT/gitRemote.sh
  • Push changes to Remote GIThub repository

  • GIT Operations tracked by GIT diff command

    Git Operation Diagramm Add. Info
    Image git_overview2.jpg NOT Found Git Storage details

  • Workspace or Working Directory
  • Git Index or Staging Area
  • Local Repository
  • Remote Repository
  • Setup GIT Status
    Image git_chapt3_img1.jpg NOT Found Note:

  • SHA == Secure Hash Algorithm


  • Run Scripts:

  • /d/GIT/gitReset.sh – Reset GIT repository
  • /d/GIT/gitDiffSetup.sh – Add file
  • Git Status Details

  • All outstanding operations are Commited
  • Local an Remote Repository are in Sync
  • The SHA value for Local And Remote is: d525f1b
  • There are 2 Commits in your Local Repository
  • First Step: Edit File test.js Verify GIT Status
    Image git_chapt3_img2.jpg NOT Found

    Run Scripts:

  • /d/GIT/gitDiff.sh – Check for Differences


  • Git Status Details

  • test.js was modified but not added to the Staging Area
  • Run git diff HEAD to check for Diffs between Working Dir and Local Repository
  • Use git add … to add a file the the Staging Area
  • Add the test.js to the Staging Area and commit the Transaction Verify GIT Status
    Image git_chapt3_img3.jpg NOT Found Run Scripts:

  • /d/GIT/gitDiff2.sh – Check for Differences


  • Git Status Details

  • git add scripts/test.js adds the file to our Staging Area
  • Run git diff HEAD to check for Diffs between Working Dir and Local Repository
  • Run git diff –staged to check for Diffs between Staging Area and Local Repository
  • Both diff commands marks the line which we have added
  • Using Branches

    Step 1: Create Default Branch with C0,C1,C2 Snapshots Add. Info
    Image git_chapt4_img1.jpg NOT Found Git Branch Details

  • Master points to C2 Snapshot
  • Image git_chapt4_img2.jpg NOT Found Run Scripts:

  • /d/GIT/gitBranchSetup.sh – Initial Script
  • Step 2: Checkout Branch iss53 Add. Info
    Image git_chapt4_img3.jpg NOT Found Git Branch Details

  • Master points to C2 Snapshot
  • iss53 points to C2 Snapshot
  • Image git_chapt4_img4.jpg NOT Found Run Scripts:

  • /d/GIT/gitBranch1.sh – Checkout iss53
  • HEAD points to iss53
  • Step 3: Implement JS code for C3 Snapshot and finally Commit Branch iss53 Add. Info
    Image git_chapt4_img5.jpg NOT Found Git Branch Details

  • Master points to C2 Snapshot
  • iss53 points to C3 Snapshot
  • Image git_chapt4_img6.jpg NOT Found >>> Edit test.js before running script !


    Run Scripts:

  • /d/GIT/gitBranch2.sh – Commit some data an branch iss53
  • HEAD points to iss53
  • Step 4: Checkout master Branch and create new hotfix branch Add. Info
    Image git_chapt4_img7.jpg NOT Found Run Scripts:

  • /d/GIT/gitBranch3.sh – Create hotfix branch
  • HEAD points to hotfix branch aka master branch
  • Step 5: Modify test.js, Implement Hotfix on Branch hotfix and Commit the data Add. Info
    Image git_chapt4_img8.jpg NOT Found Git Branch Details

  • Master points to C2 Snapshot
  • iss53 points to C3 Snapshot
  • hotfix points to C4 Snapshot
  • Image git_chapt4_img9.jpg NOT Found >>> Edit test.js before running script !

    Run Scripts:

  • /d/GIT/gitBranch4.sh – Commit some data on branch hotfix
  • HEAD points now to hotfix branch
  • Step 6: Merge master and hotfix Branch Add. Info
    Image git_chapt4_img10.jpg NOT Found Git Merge Action

  • You’ll notice the phrase “fast-forward” in that merge.
  • C4 pointed to by the branch hotfix was directly ahead of the commit C2
  • Git simply moves the pointer forward.


  • Git Branch Details

  • Master points now to C4 Snapshot
  • iss53 points to C3 Snapshot
  • hotfix points to C4 Snapshot
  • Image git_chapt4_img11.jpg NOT Found Run Scripts:

  • /d/GIT/gitBranch5.sh – Merge master and hotfix Branch
  • HEAD points now to master/hotfix branch
  • Step 7: Delete hotfix Branch and continue work on iss53 branch Add. Info
    Image git_chapt4_img12.jpg NOT Found Run Scripts:

  • /d/GIT/gitBranch6.sh – Delete hotfix Branch and checkout iss53 branch
  • HEAD points now to iss53 branch
  • Step 8: Implement JS code for C5 Snapshot and finally Commit Branch iss53 Add. Info
    Image git_chapt4_img13.jpg NOT Found Git Branch Details

  • Master points to C4 Snapshot
  • iss53 points to C5 Snapshot
  • Image git_chapt4_img14.jpg NOT Found >>> Edit test.js before running script !


    Run Scripts:

  • /d/GIT/gitBranch7.sh – Commit data on branch iss53
  • HEAD points to iss53
  • Step 9: Try to merge iss53 branch and master branch Add. Info
    Image git_chapt4_img18.jpg NOT Found Git Merge Details

  • Master points to C4 Snapshot
  • iss53 points to C5 Snapshot
  • Image git_chapt4_img15.jpg NOT Found >>> NOTE: Merge fails !!


    Run Scripts:

  • /d/GIT/gitBranch8.sh – Merge branches
  • HEAD points to master
  • Step 10: Display Diffs between common Ancestor C2 and iss53 branch and master branch Add. Info
    Image git_chapt4_img17.jpg NOT Found Diffs between Ancestor C2/C4 and C2/C5 Snapshot

  • C2 is common ancestor for C4 and C5 Snapshot
  • Running Git Mergetool

    Step 11: Run git mergetool and fix Merge Conflict
    Image git_chapt4_mergetool.jpg NOT Found
    >>> Merge tool

  • Start GIT mergetool by running: $ git mergetool
  • Left window shows the file test.js for the master branch at Snapshot C4
  • Window in the middle shows the file test.js at Snapshot C2
  • Right window shows the file test.js for the iss53 branch at Snapshot C5


  • >>> Merge Conflict Resolution

  • master and iss 53 show no real conflict
  • Comment out the tags written by the mergetool
  • Finally leave the mergetool with vim command :wqa

  • Commit the data after MERGE request

    Step 12: Commit the Changes done by GIT mergetool Add. Info
    Image git_chapt4_img19.jpg NOT Found Git Branch Details

  • Master points to C6 Snapshot
  • iss53 points to C5 Snapshot
  • Image git_chapt4_img20.jpg NOT Found >>> NOTE: test.js was corrected in step 11 by using GIT mergetool !


    Run Scripts:

  • /d/GIT/gitBranch9.sh – Commit data after MERGE reequest
  • HEAD points now to master branch
  • Branch iss 53 can NOW be deleted !
  • Rename a remote branch

    
    Rename the local branch
    $ git branch --move STUV-184_TDB STUV-188
    
    
    Save the new branch to origing
    $ git push origin --set-upstream STUV-188
    
    warning: redirecting to https://git.informatik.fh-nuernberg.de/study-monitor/pythia-the-counselor.git/
    Total 0 (delta 0), reused 0 (delta 0)
    remote:
    remote: To create a merge request for STUV-188, visit:
    remote:   https://git.informatik.fh-nuernberg.de/study-monitor/pythia-the-counselor/merge_requests/new?merge_request%5Bsource_branch%5D=STUV-188
    remote:
    To https://git.informatik.fh-nuernberg.de/study-monitor/pythia-the-counselor
     * [new branch]      STUV-188 -> STUV-188
    Branch 'STUV-188' set up to track remote branch 'STUV-188' from 'origin'.
    
    
    Delete the old branch 
    $ git push origin --delete STUV-184_TDB
    
    warning: redirecting to https://git.informatik.fh-nuernberg.de/study-monitor/pythia-the-counselor.git/
    To https://git.informatik.fh-nuernberg.de/study-monitor/pythia-the-counselor
     - [deleted]         STUV-184_TDB
    
    

    Add current changes to an NEW Branch

    Valdiate  Status: 
      git status
      On branch develop
      Your branch is ahead of 'origin/develop' by 1 commit.
       (use "git push" to publish your local commits)
    
      Changes not staged for commit:
        (use "git add ..." to update what will be committed)
        (use "git checkout -- ..." to discard changes in working directory)
    
            modified:   application/src/main/java/de/thnuernberg/in/stuv/pythia/application/config/ConfigurationService.java
    
    Add changes to new Branch 
      git branch STUV-330
      git checkout STUV-330
    
    strong>Valdiate  Status: 
    On branch STUV-330
    Changes not staged for commit:
      (use "git add ..." to update what will be committed)
      (use "git checkout -- ..." to discard changes in working directory)
    
            modified:   application/src/main/java/de/thnuernberg/in/stuv/pythia/application/config/ConfigurationService.java
    
    

    Reference

    Restoring the OCR – 12.2

    Backup currently active OCR 
    [root@dsctw21 peer]# ocrconfig -manualbackup
    dsctw21     2017/05/21 09:07:10     +MGMT:/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631     0     
    [root@dsctw21 peer]#  ocrconfig -showbackup
    PROT-24: Auto backups for the Oracle Cluster Registry are not available
    dsctw21     2017/05/21 09:07:10     +MGMT:/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631     0   
    
    Locate all OCR backups 
    ASMCMD> find --type OCRBACKUP / *
    +MGMT/dsctw/OCRBACKUP/14348721.293.944515403
    +MGMT/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631
    ASMCMD> ls -l +MGMT/dsctw/OCRBACKUP/14348721.293.944515403
    Type       Redund  Striped  Time             Sys  Name
    OCRBACKUP  UNPROT  COARSE   MAY 20 21:00:00  Y    14348721.293.944515403
    ASMCMD> ls -l +MGMT/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631
    Type       Redund  Striped  Time             Sys  Name
    OCRBACKUP  UNPROT  COARSE   MAY 21 09:00:00  Y    backup_20170521_090710.ocr.292.944557631
    
    --> Note the first backup was created by root.sh !
        After a GNS corruption we need to restore to the OCR created by root.sh
    
    List the nodes and cluster resources in your cluster by running the following command on one node:
    [grid@dsctw21 ~]$ olsnodes
    dsctw21
    dsctw22
    
    [grid@dsctw21 ~]$ crs
    *****  Cluster Resources: *****
    Resource NAME               INST   TARGET       STATE        SERVER          STATE_DETAILS
    --------------------------- ----   ------------ ------------ --------------- -----------------------------------------
    ora.LISTENER_SCAN1.lsnr        1   ONLINE       ONLINE       dsctw22         STABLE  
    ora.LISTENER_SCAN2.lsnr        1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.LISTENER_SCAN3.lsnr        1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.MGMTLSNR                   1   ONLINE       ONLINE       dsctw21         169.254.156.94 192.1 68.2.151,STABLE
    ora.asm                        1   ONLINE       ONLINE       dsctw21         Started,STABLE  
    ora.asm                        2   ONLINE       ONLINE       dsctw22         Started,STABLE  
    ora.asm                        3   OFFLINE      OFFLINE      -               STABLE  
    ora.cvu                        1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.dsctw21.vip                1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.dsctw22.vip                1   ONLINE       ONLINE       dsctw22         STABLE  
    ora.gns                        1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.gns.vip                    1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.ioserver                   1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.ioserver                   2   ONLINE       ONLINE       dsctw22         STABLE  
    ora.ioserver                   3   ONLINE       OFFLINE      -               STABLE  
    ora.mgmtdb                     1   ONLINE       ONLINE       dsctw21         Open,STABLE  
    ora.qosmserver                 1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.rhpserver                  1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.scan1.vip                  1   ONLINE       ONLINE       dsctw22         STABLE  
    ora.scan2.vip                  1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.scan3.vip                  1   ONLINE       ONLINE       dsctw21         STABLE  
    
    If OCR is located in an Oracle ASM disk group, then stop the Oracle Clusterware daemon:
    [root@dsctw21 ~]# crsctl stop crs 
    [root@dsctw22 ~]# crsctl stop crs 
      
    Start the Oracle Clusterware stack on one node in exclusive mode by running the following command as root:
    [root@dsctw21 ~]#  crsctl start crs -excl -nocrs
    
    The -nocrs option ensures that the CRSD process and OCR do not start with the rest of the Oracle Clusterware stack.
    
    Ignore any errors that display.
    Check whether CRSD is running by running the following command:
    [root@dsctw21 ~]# crsi
    
    *****  Local Resources: *****
    Resource NAME               INST   TARGET       STATE        SERVER          STATE_DETAILS
    --------------------------- ----   ------------ ------------ --------------- -----------------------------------------
    ora.asm                        1   ONLINE       ONLINE       dsctw21         Started,STABLE  
    ora.cluster_interconnect.haip  1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.crf                        1   OFFLINE      OFFLINE      -               STABLE  
    ora.crsd                       1   OFFLINE      OFFLINE      -               STABLE  
    ora.cssd                       1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.cssdmonitor                1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.ctssd                      1   ONLINE       ONLINE       dsctw21         OBSERVER,STABLE  
    ora.diskmon                    1   OFFLINE      OFFLINE      -               STABLE  
    ora.driver.afd                 1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.drivers.acfs               1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.evmd                       1   ONLINE       INTERMEDIATE dsctw21         STABLE  
    ora.gipcd                      1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.gpnpd                      1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.mdnsd                      1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.storage                    1   OFFLINE      OFFLINE      -               STABLE  
    
    
        If CRSD is running, then stop it by running the following command as root:
        # crsctl stop resource ora.crsd -init
    
       
    Locate all OCR backups 
    ASMCMD> find --type OCRBACKUP / *
    +MGMT/dsctw/OCRBACKUP/14348721.293.944515403
    +MGMT/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631
    ASMCMD> ls -l +MGMT/dsctw/OCRBACKUP/14348721.293.944515403
    Type       Redund  Striped  Time             Sys  Name
    OCRBACKUP  UNPROT  COARSE   MAY 20 21:00:00  Y    14348721.293.944515403
    ASMCMD> ls -l +MGMT/dsctw/OCRBACKUP/backup_20170521_090710.ocr.292.944557631
    Type       Redund  Striped  Time             Sys  Name
    OCRBACKUP  UNPROT  COARSE   MAY 21 09:00:00  Y    backup_20170521_090710.ocr.292.944557631
    
    Restore OCR with an OCR backup that you can identify in "Listing Backup Files" by running the following command as root:
    [root@dsctw21 ~]# ocrconfig -restore +MGMT/dsctw/OCRBACKUP/14348721.293.944515403
    
        Note:
            If the original OCR location does not exist, then you must create an empty (0 byte) OCR location before 
            you run the ocrconfig -restore command.
            Ensure that the OCR devices that you specify in the OCR configuration exist and that these OCR devices are valid.
            If you configured OCR in an Oracle ASM disk group, then ensure that the Oracle ASM disk group exists and is mounted.
            If the OCR backup file is located in an Oracle ASM disk group, then ensure that the disk group exists and is mounted.
    
    [root@dsctw21 ~]# ocrcheck
    Status of Oracle Cluster Registry is as follows :
         Version                  :          4
         Total space (kbytes)     :     409568
         Used space (kbytes)      :       3992
         Available space (kbytes) :     405576
         ID                       : 2008703361
         Device/File Name         :      +DATA
                                        Device/File integrity check succeeded
                                        Device/File not configured
                                        Device/File not configured
                                        Device/File not configured
                                        Device/File not configured
         Cluster registry integrity check succeeded
         Logical corruption check succeeded
    
    [root@dsctw21 ~]# crsctl stop crs -f
        
        Run the ocrconfig -repair -replace command as root on all the nodes in the cluster where you did not the 
        ocrconfig -restore command. For example, if you ran the ocrconfig -restore command on node 1 of a four-node 
        cluster, then you must run the ocrconfig -repair -replace command on nodes 2, 3, and 4.
    
    Begin to start Oracle Clusterware by running the following command as root on all of the nodes:
    [root@dsctw21 ~]#  crsctl start crs
    [root@dsctw22 ~]#  crsctl start crs
    
    Verify OCR integrity of all of the cluster nodes that are configured as part of your cluster by running the following CVU command:
    [grid@dsctw21 ~]$  cluvfy comp ocr -n all -verbose
    Verifying OCR Integrity ...PASSED
    Verification of OCR integrity was successful. 
    CVU operation performed:      OCR integrity
    Date:                         May 21, 2017 8:13:54 PM
    CVU home:                     /u01/app/122/grid/
    User:                         grid
    
    Verify cluster resources 
    [root@dsctw22 ~]#  crs
    *****  Cluster Resources: *****
    Resource NAME               INST   TARGET       STATE        SERVER          STATE_DETAILS
    --------------------------- ----   ------------ ------------ --------------- -----------------------------------------
    ora.LISTENER_SCAN1.lsnr        1   ONLINE       ONLINE       dsctw22         STABLE  
    ora.LISTENER_SCAN2.lsnr        1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.LISTENER_SCAN3.lsnr        1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.MGMTLSNR                   1   ONLINE       ONLINE       dsctw21         169.254.156.94 192.1 68.2.151,STABLE
    ora.asm                        1   ONLINE       ONLINE       dsctw21         Started,STABLE  
    ora.asm                        2   ONLINE       ONLINE       dsctw22         Started,STABLE  
    ora.asm                        3   OFFLINE      OFFLINE      -               STABLE  
    ora.cvu                        1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.dsctw21.vip                1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.dsctw22.vip                1   ONLINE       ONLINE       dsctw22         STABLE  
    ora.gns                        1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.gns.vip                    1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.ioserver                   1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.ioserver                   2   ONLINE       ONLINE       dsctw22         STABLE  
    ora.ioserver                   3   ONLINE       OFFLINE      -               STABLE  
    ora.mgmtdb                     1   ONLINE       ONLINE       dsctw21         Open,STABLE  
    ora.qosmserver                 1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.rhpserver                  1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.scan1.vip                  1   ONLINE       ONLINE       dsctw22         STABLE  
    ora.scan2.vip                  1   ONLINE       ONLINE       dsctw21         STABLE  
    ora.scan3.vip                  1   ONLINE       ONLINE       dsctw21         STABLE

    Cleanup a failed 12.2 GRID installation

    Cleanup GRID resources and directories

    Log in as the root user on a node where you encountered an error.
    Change directory to Grid_home/crs/install. For example:
    Run rootcrs.sh with the -deconfig and -force flags. For example:
    
    [root@dsctw21 ~]# export GRID_HOME=/u01/app/122/grid
    [root@dsctw22 ~]# $GRID_HOME/crs/install/rootcrs.sh -verbose -deconfig -force
    
    Repeat on other nodes as required.
    If you are deconfiguring Oracle Clusterware on all nodes in the cluster, then on the last node, enter the following command:
    
    [root@dsctw21 ~]# export GRID_HOME=/u01/app/122/grid
    [root@dsctw21 ~]# $GRID_HOME/crs/install/rootcrs.sh -verbose -deconfig -force -lastnode
    
    The -lastnode flag completes deconfiguration of the cluster, including the OCR and voting files.
    
    Current Settings
    ORACLE_BASE=/u01/app/grid
    ORACLE_HOME=/u01/app/122/grid
    
    Remove oraInventory Files
    [root@dsctw21 oraInventory]# rm -rf  /u01/app/oraInventory/*
    
    Remove Files from ORACLE_BASE, ORACLE_HOME
    [root@dsctw21 ~]# rm -rf /u01/app/grid/*
    [root@dsctw21 ~]# rm -rf /u01/app/122/grid/*
    
    Remove above files/directories on the 2.nd node too !
    
    [root@dsctw21 grid]# chown  grid:oinstall /u01/app/122/grid
    [root@dsctw21 grid]# chown  grid:oinstall /u01/app/grid
    [root@dsctw21 grid]# chown  grid:oinstall /u01/app/oraInventory/
    
    [root@dsctw22 oraInventory]# chown  grid:oinstall /u01/app/122/grid
    [root@dsctw22 oraInventory]# chown  grid:oinstall /u01/app/grid
    [root@dsctw22 oraInventory]# chown  grid:oinstall /u01/app/oraInventory/
    
    Verify directories
    [root@dsctw21 ~]# ls  -lasi /u01/app/122/grid
    total 0
    33595586 0 drwxr-xr-x. 2 grid oinstall  6 May  4 15:49 .
    50331840 0 drwxr-xr-x. 3 root oinstall 17 May  4 15:49 ..
    [root@dsctw22 ~]#  ls  -lasi /u01/app/122/grid
    total 0
    33595586 0 drwxr-xr-x. 2 grid oinstall  6 May  4 15:49 .
    50331840 0 drwxr-xr-x. 3 root oinstall 17 May  4 15:49 ..
    .....
    

    Cleanup our ASM Disks

    On Node 1 run 
    [root@dsctw21 ~]# ./cleanupASM_Disks.sh
    Model: ATA VBOX HARDDISK (scsi)
    Disk /dev/sdb: 21.5GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos
    Disk Flags: 
    ..
    brw-rw----. 1 root disk     8,   0 May  5 07:36 /dev/sda
    brw-rw----. 1 root disk     8,   1 May  5 07:36 /dev/sda1
    brw-rw----. 1 root disk     8,   2 May  5 07:36 /dev/sda2
    brw-rw----. 1 grid asmadmin 8,  16 May  5 08:21 /dev/sdb
    brw-rw----. 1 grid asmadmin 8,  32 May  5 08:21 /dev/sdc
    brw-rw----. 1 grid asmadmin 8,  48 May  5 08:21 /dev/sdd
    brw-rw----. 1 grid asmadmin 8,  64 May  5 08:21 /dev/sde
    brw-rw----. 1 grid asmadmin 8,  80 May  5 08:21 /dev/sdf
    brw-rw----. 1 grid asmadmin 8,  96 May  5 08:21 /dev/sdg
    brw-rw----. 1 grid asmadmin 8, 112 May  5 08:21 /dev/sdh
    brw-rw----. 1 root disk     8, 128 May  5 07:36 /dev/sdi
    brw-rw----. 1 root disk     8, 129 May  5 07:36 /dev/sdi1
    
    On Node 2 just run 
    [root@dsctw22 ~]# ./modifyASM_Disks.sh
    Model: ATA VBOX HARDDISK (scsi)
    Disk /dev/sdb: 21.5GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos
    Disk Flags: 
    ....
    brw-rw----. 1 root disk     8,   0 May  5 07:22 /dev/sda
    brw-rw----. 1 root disk     8,   1 May  5 07:22 /dev/sda1
    brw-rw----. 1 root disk     8,   2 May  5 07:22 /dev/sda2
    brw-rw----. 1 grid asmadmin 8,  16 May  5 08:25 /dev/sdb
    brw-rw----. 1 grid asmadmin 8,  32 May  5 08:25 /dev/sdc
    brw-rw----. 1 grid asmadmin 8,  48 May  5 08:25 /dev/sdd
    brw-rw----. 1 grid asmadmin 8,  64 May  5 08:25 /dev/sde
    brw-rw----. 1 grid asmadmin 8,  80 May  5 08:25 /dev/sdf
    brw-rw----. 1 grid asmadmin 8,  96 May  5 08:25 /dev/sdg
    brw-rw----. 1 grid asmadmin 8, 112 May  5 07:25 /dev/sdh
    brw-rw----. 1 root disk     8, 128 May  5 07:22 /dev/sdi
    brw-rw----. 1 root disk     8, 129 May  5 07:22 /dev/sdi1
    
    Note protections on both nodes should be identical !
    

    Scripts used in this article

    [root@dsctw21 ~]# cat  cleanupASM_Disks.sh
    parted /dev/sdb mklabel msdos
    parted /dev/sdc mklabel msdos
    parted /dev/sdd mklabel msdos
    parted /dev/sde mklabel msdos
    parted /dev/sdf mklabel msdos
    parted /dev/sdg mklabel msdos
    parted /dev/sdh mklabel msdos
    
    parted /dev/sdb print
    parted /dev/sdc print
    parted /dev/sdd print
    parted /dev/sde print
    parted /dev/sdf print
    parted /dev/sdg print
    parted /dev/sdh print
    ./modifyASM_Disks.sh
    
    [root@dsctw21 ~]# cat  modifyASM_Disks.sh
    parted /dev/sdb print
    parted /dev/sdc print
    parted /dev/sdd print
    parted /dev/sde print
    parted /dev/sdf print
    parted /dev/sdg print
    parted /dev/sdh print
    
    chmod 660 /dev/sdb
    chmod 660 /dev/sdc
    chmod 660 /dev/sdd
    chmod 660 /dev/sde
    chmod 660 /dev/sdf
    chmod 660 /dev/sdg
    chmod 660 /dev/sdh
    
    chown grid:asmadmin /dev/sdb
    chown grid:asmadmin /dev/sdc
    chown grid:asmadmin /dev/sdd
    chown grid:asmadmin /dev/sde
    chown grid:asmadmin /dev/sdf
    chown grid:asmadmin /dev/sdg
    chown grid:asmadmin /dev/sdh
    
    ls -l /dev/sd*

    Convert a administrator managed RAC database to policy managed [ 12.2 ]

    Check Database and Pool Status

    Check RAC database status
    [grid@ractw21 ~]$   srvctl config database -d ractw
    Database unique name: ractw
    Database name: ractw
    Oracle home: /u01/app/oracle/product/122/ractw2
    Oracle user: oracle
    Spfile: +DATA/RACTW/PARAMETERFILE/spfile.325.941894301
    Password file: +DATA/RACTW/PASSWORD/pwdractw.284.941893247
    Domain: example.com
    Start options: open
    Stop options: immediate
    Database role: PRIMARY
    Management policy: AUTOMATIC
    Server pools: 
    Disk Groups: DATA
    Mount point paths: 
    Services: 
    Type: RAC
    Start concurrency: 
    Stop concurrency: 
    OSDBA group: dba
    OSOPER group: 
    Database instances: ractw1,ractw2
    Configured nodes: ractw21,ractw22
    CSS critical: no
    CPU count: 0
    Memory target: 0
    Maximum memory: 0
    Default network number for database services: 
    Database is administrator managed
    
    Check pool status 
    [oracle@ractw21 ~]$  srvctl status srvpool -a
    Server pool name: Free
    Active servers count: 0
    Active server names: 
    Server pool name: Generic
    Active servers count: 2
    Active server names: ractw21,ractw22
    NAME=ractw21 STATE=ONLINE
    NAME=ractw22 STATE=ONLINE
    

    Convert Database to Policy Managed

    Stop database,add RAC database to pool and finally check pool status [grid@ractw21 grid]$ srvctl stop database -d ractw
    [grid@ractw21 grid]$ srvctl add srvpool -g TopPriority  -l 1 -u 2 -i 5
    [grid@ractw21 grid]$ srvctl modify database -d ractw -g TopPriority
                            
    PRCD-1130 : Failed to convert administrator-managed database ractw into a policy-managed database to use server pool TopPriority
    PRCR-1071 : Failed to register or update resource ora.ractw.db
    CRS-0245:  User doesn't have enough privilege to perform the operation
    
    Run as user Oracle :
    [oracle@ractw21 ~]$ srvctl modify database -d ractw -g TopPriority
    
    [grid@ractw21 grid]$ srvctl config  srvpool -g TopPriority
    Server pool name: TopPriority
    Importance: 5, Min: 1, Max: 2
    Category: hub
    Candidate server names: 
    
    [grid@ractw21 grid]$  srvctl config  database -d ractw
    Database unique name: ractw
    Database name: ractw
    Oracle home: /u01/app/oracle/product/122/ractw2
    Oracle user: oracle
    Spfile: +DATA/RACTW/PARAMETERFILE/spfile.325.941894301
    Password file: +DATA/RACTW/PASSWORD/pwdractw.284.941893247
    Domain: example.com
    Start options: open
    Stop options: immediate
    Database role: PRIMARY
    Management policy: AUTOMATIC
    Server pools: TopPriority
    Disk Groups: DATA
    Mount point paths: 
    Services: 
    Type: RAC
    Start concurrency: 
    Stop concurrency: 
    OSDBA group: dba
    OSOPER group: 
    Database instances: 
    Configured nodes: 
    CSS critical: no
    CPU count: 0
    Memory target: 0
    Maximum memory: 0
    Default network number for database services: 
    Database is policy managed
    
    [oracle@ractw21 ~]$ srvctl start database -d ractw
    PRCR-1079 : Failed to start resource ora.ractw.db
    CRS-2643: The server pool(s) where resource 'ora.ractw.db' could run have no servers
    
    Remove category and add servers 
    
    Trying to add servers 
    [grid@ractw21 ~]$ srvctl modify srvpool -serverpool TopPriority  -servers  "ractw21,ractw22" -verbose
    [grid@ractw21 ~]$ srvctl config srvpool -serverpool TopPriority 
    Server pool name: TopPriority
    Importance: 5, Min: 1, Max: 2
    Category: 
    Candidate server names: ractw21,ractw22
    
    [grid@ractw21 ~]$  srvctl status srvpool -a
    Server pool name: Free
    Active servers count: 0
    Active server names: 
    Server pool name: Generic
    Active servers count: 2
    Active server names: ractw21,ractw22
    NAME=ractw21 STATE=ONLINE
    NAME=ractw22 STATE=ONLINE
    Server pool name: TopPriority
    Active servers count: 0
    Active server names: 
    
    Both severs  ractw21,ractw22 still assigned to GENERIC pool !
    
    Removing our admin managed test database ! 
    [oracle@ractw21 ~]$ srvctl remove database -db test
    Remove the database test? (y/[n]) y
    

    Verify Database Status and Pool Status after changes

    Now verify pool 
    [grid@ractw21 ~]$  srvctl status srvpool -a
    Server pool name: Free
    Active servers count: 0
    Active server names: 
    Server pool name: Generic
    Active servers count: 0
    Active server names: 
    Server pool name: TopPriority
    Active servers count: 2
    Active server names: ractw21,ractw22
    NAME=ractw21 STATE=ONLINE
    NAME=ractw22 STATE=ONLINE
    --> Now both servers are assigned to our TopPriority pool 
    
    Starting ractw database and confirm cluster resource status
    [oracle@ractw21 dbs]$ srvctl start  database -db ractw
    
    *****  Cluster Resources: *****
    Resource NAME               INST   TARGET    STATE        SERVER          STATE_DETAILS
    --------------------------- ----   ------------ ------------ --------------- -----------------------------------------
    ora.ractw.db                   1   ONLINE    ONLINE       ractw21         Open,HOME=/u01/app/oracle/product/122/ractw2,STABLE
    ora.ractw.db                   2   ONLINE    ONLINE       ractw22         Open,HOME=/u01/app/oracle/product/122/ractw2,STABLE

    Installing Dtrace on OEL 7.3

     # yum install dtrace-modules-4.1.12-61.1.33.el7uek.x86_64
     # yum install dtrace-modules-provider-headers.x86_64 0.5.3-2.el7     
     # yum install libdtrace-ctf-devel.x86_64 
    
    [root@ractw21 OEL73]# rpm -ihv dtrace-utils-devel-0.5.1-3.el7.x86_64.rpm  dtrace-utils-0.5.1-3.el7.x86_64.rpm 
    Preparing...                          ################################# [100%]
    Updating / installing...
       1:dtrace-utils-0.5.1-3.el7         ################################# [ 50%]
       2:dtrace-utils-devel-0.5.1-3.el7   ################################# [100%]
    
    [root@ractw21 OEL73]# dtrace -l
       ID   PROVIDER            MODULE                          FUNCTION NAME
        1     dtrace                                                     BEGIN
        2     dtrace                                                     END
        3     dtrace                                                     ERROR
    [root@ractw21 OEL73]# modprobe systrace
    [root@ractw21 OEL73]# dtrace -l | more
       ID   PROVIDER            MODULE                          FUNCTION NAME
        1     dtrace                                                     BEGIN
        2     dtrace                                                     END
        3     dtrace                                                     ERROR
        4    syscall           vmlinux                              read entry
        5    syscall           vmlinux                              read return
        6    syscall           vmlinux                             write entry
        7    syscall           vmlinux                             write return
        8    syscall           vmlinux                              open entry
    ....
    

    Reference

    Using Recursions and Closures

    • Note: Use setTimeout to avoid Error Call Stack Size Exceeded
    • JavaScript Code:
    cPanel.scheduleMapsSliderStatus = function (rc, func) {
        var runCount = rc;
        var funcName = 'scheduleMapsSliderStatus()';
        var logHeader = myControlPanel.getLogHeaderCF(funcName, func);
    
        var slideShowStatusSummaryField =  myObjects.getPanel("slideShowStatusSummaryField");
        var bgColor = slideShowStatusSummaryField.css('background-color');
        var fgColor =  slideShowStatusSummaryField.css('color');
        
        function MapsSliderStatusClosure() {
            var mesgFlag = runCount % 2;
            var panelMode = myControlPanel.getPanelMode(funcName);
            var slideShowStatusSummaryField = null;
            if (mesgFlag === 0) {
                updateSliderStatus("Click on PLAY button to start Animation! ");
                    // Need to reread slideShowStatusSummaryField Objects as this is recreated by  updateSliderStatus()
                slideShowStatusSummaryField =  myObjects.getPanel("slideShowStatusSummaryField");
                slideShowStatusSummaryField.css('background-color', bgColor);
    
            } else {
                updateSliderStatus("Click on GREEN Marker to start Slideshow !");
                    // Need to reread slideShowStatusSummaryField Objects as this is recreated by  updateSliderStatus()
                slideShowStatusSummaryField =  myObjects.getPanel("slideShowStatusSummaryField");
                slideShowStatusSummaryField.css('background-color', 'green');
            }
            if (runCount === 10 ) {   // run this Animation ONLY 10x
                return;
             }
            if (panelMode === "maps") { // Don't schedule again if panelMode !='maps'
                runCount++;
                setTimeout(MapsSliderStatusClosure, 2000, runCount, funcName)
            }
        };
        MapsSliderStatusClosure();   // Just all the closure function a first time
    };

     

     

    Copy a Maven based JPA project to new project folder

    New Project Details

    ArtifactId                   : WFJPA2PC
    New Project Home             : /home/oracle/NetBeansProjects/GIT/WFJPA2PC 
    New persistence.xml location :  /home/oracle/NetBeansProjects/GIT/WFJPA2PC/src/main/resources/META-INF/persistence.xml Your old and working project should have a working pom.xml and persistence.xml 

    Modify and copy pom,xml

    Modify artifactId and name in your pom.xml 
        <artifactId>WFJPA2PC</artifactId>
        <name>WFJPA2PC</name>
    
    Copy pom.xml to your new Project Folder 
    $ cp pom.xml   /home/oracle/NetBeansProjects/GIT/WFJPA2PC
    
    Add JSF support to our project 
    Project -> Properties -> Framework -> Add JavaServerFaces 
                                       -> Components -> PrimeFaces 
    

    Create new Entity Classes

    Create new Entity Classes as needed [ in our sample we use Entity Class : Accounts 
    
    Source package -> New -> EnititY Class form Database 
      -> Select Datasource 
      -> Select Table Name : Accounts  
      -> Add 
      -> Finish 
    

    Create a new JPA persistence.xml and copy over the working persistence.xml

    Netbeans : File -> NEW ->  Persistence Unit 
    
    Locate your Persistence Location of persistence.xml:
      /home/oracle/NetBeansProjects/GIT/WFJPA2PC/src/main/resources/META-INF/persistence.xml
    
    -> Copy over your new RAC XA persistence.xml to   src/main/resources/META-INF/persistence.xml
       and add your newly created Entitly Classes    
    
    Sample for a Hibernate persistence.xml with Entity Class : Accounts
    <?xml version="1.0"?>
    <persistence xmlns="http://java.sun.com/xml/ns/persistence"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"
                 version="2.0">
    
        <persistence-unit name="RacBankAHibPU" transaction-type="JTA">
            <provider>org.hibernate.ejb.HibernatePersistence</provider>
            <jta-data-source>java:/jboss/datasources/xa_rac12g_banka</jta-data-source>
            <class>com.hhu.wfjpa2pc.Accounts</class>
            <properties>
                <property name="hibernate.transaction.jta.platform"
                     value="org.hibernate.service.jta.platform.internal.JBossAppServerJtaPlatform" />
                <property name="hibernate.show_sql" value="true" />
            </properties>
    </persistence>

    Connection timeout message when running nslookup against GNS SCAN Address

    Nslookup Error
    [root@ns1 named]#  nslookup grac4-scan.grid4.example.com
    ;; connection timed out; trying next origin
    Server:        192.168.5.50
    Address:    192.168.5.50#53
    ** server can't find grac4-scan.grid4.example.com: NXDOMAIN
    
    /var/log/messages of the BIND Nameserver  contains  a lot of IPV6 related errors 
      using GNS host : GNSTESTHOST 
    ..
    Jun 24 15:19:08 ns1 named[7529]: error (network unreachable) resolving 'GNSTESTHOST.grid4.example.com.de.oracle.com/AAAA/IN': 202.12.27.33#53
    Jun 24 15:19:08 ns1 named[7529]: error (network unreachable) resolving 'GNSTESTHOST.grid4.example.com.de.oracle.com/A/IN': 2001:503:ba3e::2:30#53
    Jun 24 15:19:08 ns1 named[7529]: error (network unreachable) resolving 'GNSTESTHOST.grid4.example.com.de.oracle.com/AAAA/IN': 2001:503:ba3e::2:30#53
    Jun 24 15:19:08 ns1 named[7529]: error (network unreachable) resolving 'GNSTESTHOST.grid4.example.com.de.oracle.com/A/IN': 2001:500:2::c#53
    Jun 24 15:19:08 ns1 named[7529]: error (network unreachable) resolving 'GNSTESTHOST.grid4.example.com.de.oracle.com/AAAA/IN': 2001:500:2::c#53
    Jun 24 15:19:08 ns1 named[7529]: error (network unreachable) resolving 'GNSTESTHOST.grid4.example.com.de.oracle.com/A/IN': 2001:500:2d::d#53
    Jun 24 15:19:08 ns1 named[7529]: error (network unreachable) resolving 'GNSTESTHOST.grid4.example.com.de.oracle.com/AAAA/IN': 2001:500:2d::d#53
    ....
    Jun 24 15:19:08 ns1 named[7529]: error (network unreachable) resolving 'GNSTESTHOST.grid4.example.com.de.oracle.com/A/IN': 2001:dc3::35#53
    Jun 24 15:19:08 ns1 named[7529]: error (network unreachable) resolving 'GNSTESTHOST.grid4.example.com.de.oracle.com/AAAA/IN': 2001:dc3::35#53
    Jun 24 15:19:18 ns1 named[7529]: error (network unreachable) resolving 'GNSTESTHOST.grid4.example.com.de.oracle.com/A/IN': 2001:502:f3ff::64#53
    
    -> All the failed IVPV6 requests add a delay for the request and leads to the nslookup Timeout
    -> As our network can't handle IPV6 we need to disable IVP6. 
    
    FIX : Disable ipv6 on CentOS 6/7 / RHEL 6/7 
    
    Edit the /etc/sysctl.conf.
    # vi /etc/sysctl.conf
    Put the following entry to disable IPv6 for all adapter.
    net.ipv6.conf.all.disable_ipv6 = 1
    
    [root@ns1 named]#  ifconfig eth1
    eth1      Link encap:Ethernet  HWaddr 08:00:27:2B:5E:DD
              inet addr:192.168.5.50  Bcast:192.168.5.255  Mask:255.255.255.0
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:2171 errors:0 dropped:0 overruns:0 frame:0
              TX packets:1461 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:159819 (156.0 KiB)  TX bytes:185391 (181.0 KiB)
    -> No IPV6 reference anymore 
    
    After disabling IPV6 only very less named records should be logged in /var/log/messages : 
    Jun 24 15:49:07 ns1 named[3152]: zone 2.168.192.in-addr.arpa/IN: sending notifies (serial 2009011201)
    Jun 24 15:49:07 ns1 named[3152]: zone 5.168.192.in-addr.arpa/IN: sending notifies (serial 2009011201)
    Jun 24 15:50:45 ns1 named[3152]: error (FORMERR) resolving 'GNSTESTHOST.grid4.example.com/AAAA/IN': 192.168.5.54#53
    Jun 24 16:00:48 ns1 named[3152]: error (FORMERR) resolving 'GNSTESTHOST.grid4.example.com/AAAA/IN': 192.168.5.54#53
    Jun 24 16:10:50 ns1 named[3152]: error (FORMERR) resolving 'GNSTESTHOST.grid4.example.com/AAAA/IN': 192.168.5.54#53
    Jun 24 16:16:52 ns1 named[3152]: client 192.168.5.101#29873: RFC 1918 response from Internet for 5.1.168.192.in-addr.arpa
    ...
    
    Nslookup should now  work fine 
    [root@ns1 log]#  nslookup grac4-scan.grid4.example.com
    Server:        192.168.5.50
    Address:    192.168.5.50#53
    
    Non-authoritative answer:
    Name:    grac4-scan.grid4.example.com
    Address: 192.168.5.200
    Name:    grac4-scan.grid4.example.com
    Address: 192.168.5.165
    Name:    grac4-scan.grid4.example.com
    Address: 192.168.5.166