Consumer related softwares

Paid versions Gravity write - AI driven for story generation 11Eleven labs - AI driven voice genereation for stories Adobe Express - for animation capcut, Clickchamp - for final vedio clips merge

Splunk and Sterling Integrator Integration

 Splunk Installation

rpm -i --prefix=/mnt/splunk2 splunk-8.2.2-87344edfcdb4-linux-2.6-x86_64.rpm

./splunk start --accept-license

Create the admin user - splunkadmin

Splunk DB Connect Manual Configuration

 

You can also install DB Connect by copying its directory into your Splunk Enterprise apps directory:

Download Splunk DB Connect and save it to a temporary location that you can access from your Splunk Enterprise instance.

Un-tar the download.

Move the splunk_app_db_connect directory into $SPLUNK_HOME/etc/apps.

Restart Splunk Enterprise (./mnt/splunk2/splunk/bin/splunk stop and then ./mnt/splunk2/splunk/bin/splunk start)

After you navigate to the Splunk DB Connect app, You may get "Cannot communicate with task server, please check your settings"

Setup the JRE Installation Path(JAVA_HOME) in the UI - 

Check If java is install by issuing java -version command. If not installed, download the latest tar.gz (jdk-8u202-linux-i586.tar.gz) package from Oracle java.

tar zxvf jdk-8u202-linux-i586.tar.gz

Change the permissions if required - chown -R root:root jdk1.8.0_202

Navigate to the JRE, notedown the path and enter it in the UI - /mnt/jdk1.8.0_202/jre

Download the corresponding driver. Here it is DB2.

Untar the file to see the folder - jdbc_sqlj

Copy or move the db2jcc4.jar file to the $SPLUNK_HOME/etc/apps/splunk_app_db_connect/drivers directory

Uninstall Splunk

rpm -qa | grep -i splunk

Then put the command rpm -e above_command_result

Example : rpm -e splunkforwarder-6.3.2-aaff59bb082c.x86_64

Splunk Data Retreival from database

Create a read only user on the database and record the identity credentials.

Navigate to Splunk UI->apps->splunk_app_db_connect->Configuration->Identities. Create the identity using the identity credentails of the read only user

Create the Connection

On the Data Lab -> Retrieve the data. Use cron expressions for Execution Frequency like for every 2 minutes polling  - */2 * * * *

Splunk Reports and Dashboards

To get the users created

index="edi" source="YFS_PERSON_INFO" sourcetype="database"  | transaction LAST_NAME, CREATETS, MODIFYTS  |  table FIRSTNAME, LAST_NAME, EMAILID, DEPARTMENT, CREATETS, MODIFYTS |   sort by CREATETS desc

MQ Installation

1. Create a group before the installation: 

For example to create a group named mqm with GID of 1010 you would type: groupadd -g 1010 mqm 

2. Create the mqm user and add the user to mqm group

useradd -u 1001 -g mqm -G mqm, root mqm 

3. sudo passwd mqm 

4. Add the below line to get sudo access.

vi /etc/sudoers

mqm     ALL=(ALL)       ALL

4. Check the maximum open files - sysctl fs.file-max 

If value is lesser than the below add the line in /etc/sysctl.conf fs.file-max = 524288 

5. Increase the file descriptors (from root user) 

vi /etc/security/limits.conf #add the below line

mqm hard nofile 10240 

mqm soft nofile 10240 

6. gunzip the tar file and then run. sudo tar -xvf IBM_MQ_9.1.5_LINUX_X86-64.tar.gz 

7.  You must be a root to accept the lincese. Please switch to root.

8. Change directory to MQServer and then Accept the license sudo ./mqlicense.sh 

9. sudo rpm -ivh MQ*.rpm

How to keep the IP address on linux on VMware

Good file looking like this /etc/sysconfig/network-scripts/ifcfg-eth0 TYPE="Ethernet" PROXY_METHOD="none" BROWSER_ONLY="no" BOOTPROTO="dhcp" DEFROUTE="yes" IPV4_FAILURE_FATAL="no" IPV6INIT="yes" IPV6_AUTOCONF="yes" IPV6_DEFROUTE="yes" IPV6_FAILURE_FATAL="no" IPV6_ADDR_GEN_MODE="stable-privacy" NAME="enp0s3" UUID="43558b3d-8a5d-4555-b105-c61eb883b0b1" DEVICE="enp0s3" ONBOOT="yes" ********************************************************* One more possibility /etc/sysconfig/network-scripts/ifcfg-(interface-name) To find your network interface name, you can use the following # nmcli con # vim /etc/sysconfig/network-scripts/ifcfg-enp0s3 TYPE="Ethernet" BOOTPROTO="none" NAME="enp0s3" IPADDR="192.168.20.150" NETMASK="255.255.255.0" GATEWAY="192.168.20.1" DEVICE="enp0s3" ONBOOT="yes" Then restart the NetworkManager with: # systemctl restart NetworkManager

Step by step process for installing Aspera Fasp.io gateway

Step by step process for installing Aspera Fasp.io gateway Follow the IBM page. I got all these steps from there - https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.1.0/com.ibm.mq.con.doc/aspera_def_unixlinux.htm Download ASP_FASP.IO_GW_V1.0.1_LNX_EN.zip from IBM site and keep it available in the server Example: /opt/softwares/Aspera/ Unzip the file. Example: sudo unzip ASP_FASP.IO_GW_V1.0.1_LNX_EN.zip Use rpm file with yum install. Example: sudo yum install ./ibm-fasp.io-gateway-1.0.1-1.x86_64.rpm ........ ....... Installed size: 8.8 M Is this ok [y/d/N]: y Then it looks like the below. Running transaction test Transaction test succeeded Running transaction Installing : ibm-fasp.io-gateway-1.0.1-1.x86_64 1/1 Verifying : ibm-fasp.io-gateway-1.0.1-1.x86_64 1/1 Installed: ibm-fasp.io-gateway.x86_64 0:1.0.1-1 Complete! _________________________________________________________________________________________ Location of installed files: Executable: /usr/bin Default configuration files: /etc/fasp.io Documentation: /usr/share/fasp.io-gateway SystemD startup file: /etc/systemd/system/fasp.io-gateway.service ************************************************************************************************** Copied the below example from IBM Site only Example This example defines an Aspera gateway connection on two machines running Linux. The configuration is as follows: The IP address of the local gateway machine is 9.20.193.107. The IP address of the remote gateway machine is 9.20.192.115. The local queue manager is running on a machine with IP address 9.20.121.5. The remote queue manager is running on a machine with IP address 9.20.121.25. Both queue managers are listening on port 1414. The queue manager channel on the local queue manager is changed to connect to the local Aspera gateway using conname 9.20.193.107(1500). The queue manager channel on the remote queue manager is changed to connect to the remote Aspera gateway using conname 9.20.192.115(1500). Define an Aspera gateway connection on the local gateway machine: Install the Aspera gateway: rpm -ivh ibm-fasp.io-gateway-1.0.0_qa_48-1.x86_64.rpm Modify the gateway.toml file in the /etc/fasp.io directory that was created by the install. Edit it to set the local gateway definitions. [[bridge]] name = "Outbound" [bridge.local] protocol = "tcp" host = "9.20.193.107" port = 1500 [bridge.forward] protocol = "fasp" host = "9.20.192.115" port = 1600 [[bridge]] name = "Inbound" [bridge.local] protocol = "fasp" host = "9.20.193.107" port = 1600 [bridge.forward] protocol = "tcp" host = "9.20.121.5" port = 1414 Repeat the previous step to define an Aspera gateway connection on the remote gateway machine. Modify the gateway.toml file in the /etc/fasp.io directory that was created by the install. Edit it to set the remote gateway definitions: [[bridge]] name = "Outbound" [bridge.local] protocol = "tcp" host = "9.20.192.115" port = 1500 [bridge.forward] protocol = "fasp" host = "9.20.193.107" port = 1600 [[bridge]] name = "Inbound" [bridge.local] protocol = "fasp" host = "9.20.192.115" port = 1600 [bridge.forward] protocol = "tcp" host = "9.20.121.25" port = 1414 At each end of the connection, change the channel definition to connect to the port that the local gateway is listening on. Change the queue manager channel on the local queue manager to connect to the local Aspera gateway using conname 9.20.193.107(1500). Change the queue manager channel on the remote queue manager to connect to the remote Aspera gateway using conname 9.20.192.115(1500). Start the local gateway by running the following command on the local gateway machine: systemctl start fasp.io-gatewayCopy code Start the remote gateway by running the following command on the remote gateway machine: systemctl start fasp.io-gatewayCopy code Restart the channels. ************************************************************************************************************* In case if Queue manager and FASP gateway are on the same server. The configuration may look like below. Here local queue manager and FASP are on the same machine - 192.168.1.107. Similarly the remote FASP gateway is 192.168.1.108. [[bridge]] name = "Outbound" #connection from the local queue manager to the local fasp gateway [bridge.local] protocol = "tcp" host = "192.168.1.107" port = 1500 #from the local fasp gateway to the remote fasp gateway [bridge.forward] protocol = "fasp" host = "192.168.1.108" port = 1600 [[bridge]] name = "Inbound" #the connection from the remote fasp gateway to the local fasp gateway [bridge.local] protocol = "fasp" host = "192.168.1.107" port = 1600 #from the local gateway to the local queue manager. [bridge.forward] protocol = "tcp" host = "192.168.1.107" port = 1414 [[bridge]] name = "Outbound" [bridge.local] protocol = "tcp" host = "192.168.1.108" port = 1500 [bridge.forward] protocol = "fasp" host = "192.168.1.107" port = 1600 [[bridge]] name = "Inbound" [bridge.local] protocol = "fasp" host = "192.168.1.108" port = 1600 [bridge.forward] protocol = "tcp" host = "192.168.1.108" port = 1414

Linux setup on VMware

Setup Bridged Network adapter Select both boot and dvd iso files. rhel-server-7.7-x86_64-boot.iso rhel-server-7.7-x86_64-dvd.iso Select "Software Selection" select "Server with GUI" and other options

Select "Network & Hostname" Apply the changes

Stop firewall rules systemctl disable firewalld 

If not selected during installation then follow below

Type the following command 

to edit /etc/hostname using vi text 

editor: sudo vi /etc/hostname Delete the old name and setup new name. 

Example: cat /etc/hostname statebank 

Next Edit the /etc/hosts file: sudo nano /etc/hosts 

Replace any occurrence of the existing computer name with your new one 

 Example: cat /etc/hosts 127.0.0.1 statebank localhost.localdomain localhost4 localhost4.localdomain4 ::1 statebank localhost.localdomain localhost6 localhost6.localdomain6 

 Reboot the system to changes take effect: sudo 

reboot Good file looking like this 

/etc/sysconfig/network-scripts/ifcfg-eth0 TYPE="Ethernet" PROXY_METHOD="none" BROWSER_ONLY="no" BOOTPROTO="dhcp" DEFROUTE="yes" IPV4_FAILURE_FATAL="no" IPV6INIT="yes" IPV6_AUTOCONF="yes" IPV6_DEFROUTE="yes" IPV6_FAILURE_FATAL="no" IPV6_ADDR_GEN_MODE="stable-privacy" NAME="enp0s3" UUID="43558b3d-8a5d-4555-b105-c61eb883b0b1" DEVICE="enp0s3" ONBOOT="yes"

Apache Products Useful for Integrations

Hi All,

Please find the descriptions for Apache products which can be suitable for B2B Integrations. I copied this information from web.

MFT
1. Apache ActiveMQ - is a fast and powerful Message Broker which supports many Cross Language Clients and Protocols and many advanced features while fully supporting JMS 1.1 and J2EE 1.4.
37. The Apache FtpServer application is a 100% pure Java FTP server
41. Apache Guacamole is a clientless remote desktop gateway which supports standard protocols like VNC, RDP, and SSH.
49. The Apache HTTP Server is an open-source HTTP server for modern operating systems including UNIX, Microsoft Windows, Mac OS/X and Netware
52. The Apache Java Enterprise Mail Server (a.k.a. Apache James) is a 100% pure Java SMTP and POP3 Mail server and NNTP News server
Translators
5. Apache Anakia is an XML transformation tool that uses JDOM and Velocity to transform XML documents into the format of your choice. It provides an alternative to using Ant's style task and XSL to process XML files.
19. Apache Daffodil is an open-source implementation of the DFDL specification that uses DFDL data descriptions to parse fixed format data into an infoset, which is most commonly represented as either XML or JSON
33. Apache FOP (Formatting Objects Processor) is the world's first print formatter driven by XSL formatting objects (XSL-FO) and the world's first output independent formatter.
34. Apache Forrest™ software is a publishing framework that transforms input from various sources into a unified presentation in one or more output formats
Business Processes
3. Apache Airflow: workflow automation and scheduling that can be used to author and manage data pipelines
11. Apache Camel is a powerful open source integration library based on Enterprise Integration Patterns. Rules for Camel's routing engine can be defined in either a Java based DSL or XML
API
7. Apache APISIX is a cloud-native microservices API gateway, delivering the ultimate performance, security,open source and scalable platform for all your APIs and microservices
18. Apache CXF is an open source services framework.
Analytics
8. Apache Arrow Powering Columnar In-Memory Analytics
10. Apache Bahir Extensions to distributed analytic platforms such as Apache Spark
26. Apache Eagle open source analytics solution for identifying security and performance issues instantly on big data platforms
39. Apache Gobblin is a distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
43. Apache Hama is an efficient and scalable general-purpose BSP computing engine which can be used to speed up a large variety of compute-intensive analytics applications.
50. Apache Ignite In-Memory Database and Caching Platform is designed to deliver uncompromised performance for a wide set of in-memory computing use cases from high performance computing, to the industry most advanced data grid, in-memory SQL, in-memory file system, streaming, and more
56. Apache Lens provides an Unified Analytics interface
Database
9. Apache AsterixDB open source Big Data Management System
16. Apache CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents with your web browser, via HTTP
21. Apache Derby is an open source relational database implemented entirely in Java
25. Apache Druid analytical database software
28. Apache Empire-db is a relational database abstraction layer that allows developers to take a much more SQL-centric approach in application development than traditional Object/Relational mapping frameworks (ORM)
Operations
4. Apache Allura is an open source implementation of a software "forge", a web site that manages source code repositories, bug reports, discussions, wiki pages, blogs and more for any number of individual projects.
2. Apache Airavata: is a micro-service architecture based software framework for executing and managing computational jobs and workflows on distributed computing resources including local clusters, supercomputers, national grids, academic and commercial clouds. Airavata is dominantly used to build Web-based science gateways and assist to compose, manage, execute, and monitor large scale applications (wrapped as Web services) and workflows composed of these services.
6. Apache Apex is a unified platform for big data stream and batch processing. Use cases include ingestion, ETL, real-time analytics, alerts and real-time actions.
12. Apache Cassandra Linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure make Apache Cassandra the perfect platform for mission-critical data
13. Apache Cayenne is a powerful, full-featured, opensource framework created for developers working with relational databases. it seamlessly maps any relational database to Java objects, reducing development time and adding considerable functionality to any application which requires a database.
14. Apache Chainsaw is a GUI log viewer
15. Apache Click is a modern Java web application framework, providing a natural rich client style programming model
17. Apache Curator is a Java/JVM client library for Apache ZooKeeper, a distributed coordination service
20. DataSketches is an open source, high-performance library of stochastic streaming algorithms commonly called "sketches" in the data sciences
22. Apache DirectMemory is a off-heap cache for the Java Virtual Machine
23. ApacheDS is an extensible and embeddable directory server entirely written in Java, which has been certified LDAPv3 compatible by the Open Group
24. Apache Directory Studio is a complete directory tooling platform intended to be used with any LDAP server however it is particularly designed for use with ApacheDS
27. Apache ECharts is a charting and data visualization library written in JavaScript.
29. Apache Felix OSGi framework implementation and related technologies.
30. Apache Flex® is a highly productive, open source application framework for building and maintaining expressive web applications that deploy consistently on all major browsers, desktops and devices
31. Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store
32. Apache Fluo is a distributed processing system that lets users make incremental updates to large data sets.
35. Apache Fortress is a standards-based access management system, written in Java, that provides role-based access control, delegated administration and password policy services with LDAP.
36. Apache FreeMarker is a template engine: a Java library to generate text output (HTML web pages, e-mails, configuration files, source code, etc.) based on templates and changing data
38. Apache Giraph is an iterative graph processing system built for high scalability.
40. Apache Groovy is a powerful, optionally typed and dynamic language, with static-typing and static compilation capabilities, for the Java platform aimed at improving developer productivity thanks to a concise, familiar and easy to learn syntax
42. Apache Hadoop Distributed computing platform
44. Apache HBase software when you need random, realtime read/write access to your Big Data.
45. Apache Helix is a generic cluster management framework used for the automatic management of partitioned, replicated and distributed resources hosted on a cluster of nodes.
46. The Apache Hive (TM) data warehouse software facilitates querying and managing large datasets residing in distributed storage
47. The Apache Hivemall is a library for machine learning implemented as Hive UDFs/UDAFs/UDTFs
48. The Apache HiveMind is a framework for creating applications, not an application, or even an application server, itself.
51. Apache Isis is a framework for rapidly developing domain-driven apps in Java.
53. A single Kafka broker can handle hundreds of megabytes of reads and writes per second from thousands of clients.
54. Apache Kibble is a suite of tools for collecting, aggregating and visualizing activity in software projects.
55. Apache Labs A place for innovation where committers of the foundation can experiment with new ideas
57. Apache Lenya is an Open Source Java/XML Content Management Framework
58. Apache Log4j 2
59. Apache Lucene is a high-performance, full-featured text search engine library written entirely in Java
60. Apache MADlib Scalable, Big Data, SQL-driven machine learning framework for Data Scientistsava