Hardware requirements

Olga Semme
IMPORTANT

The following system requirements are minimal. The target sizing should be calculated based on the customer requirements.

It is preferable to use physical servers, but it is necessary to analyze the power consumption of the equipment you are considering.

Arenadata Streaming works equally effectively both on bare metal and in the cloud.

When providing network availability between servers, the following scenarios are also possible:

  • heterogeneous IT infrastructure;

  • Multi-clouds;

  • distributed installation on different infrastructures.

The sequence of preparation of physical and virtual hosts is given in the article Prepare hosts.

Examples of requirements for different services depending on the load pattern are listed below.

ZooKeeper, Kafka Manager, Schema Registry

  • Minimal

  • Proof of concept

  • Commodity

  • Balanced

Storage CPU RAM Network

400 GB SSD for OS

500 GB SSD for Data

8 cores

32 GB

1 x 10 GbE
Number of nodes Storage CPU RAM Network

3 or more

1 x 500 GB HDD for OS

1 x 2.4 TB SSD for Transaction Logs

1 x 1 TB HDD for Data

8 cores

32 GB

1 x 10 GbE
Number of nodes Storage CPU RAM Network

3 or more

2 x 500 GB HDD RAID1 for OS

2 x 2.4 TB SSD RAID1 for Transaction Logs

2 x 1 TB HDD RAID1 for Data

8 cores

32 GB

2 x 10 GbE
Number of nodes Storage CPU RAM Network

3 or more

2 x 500 GB HDD RAID1 for OS

2 x 2.4 TB SSD RAID1 for Transaction Logs

2 x 1 TB HDD RAID1 for Data

8 cores

32 GB

2 x 10 GbE
IMPORTANT
For guaranteed HA (High Availability), it is necessary to have an odd number of ZooKeeper Server instances.

Kafka Broker

  • Proof of concept

  • Commodity

  • Balanced

  • Compute-intensive

Number of nodes Storage CPU RAM Network

3 or more

1 x 500 GB HDD for OS

2 x 500 GB SSD RAID1 for Data

8 cores

32 GB

2 x 10 GbE
Number of nodes Storage CPU RAM Network

3 or more

2 x 500 GB HDD RAID1 for OS

4 x 500 GB SAS12G SSD RAID10 (PCIe Gen3 x4+) for Data

8 cores

32 GB

2 x 25 GbE
Number of nodes Storage CPU RAM Network

3 or more

2 x 500 GB HDD RAID1 for OS

8 x 500 GB SAS12G SSD RAID10 (PCIe Gen3 x8) for Data

16 cores

64 GB

2 x 25 GbE
Number of nodes Storage CPU RAM Network

6 or more

2 x 500 GB HDD RAID1 for OS

8 x 500 GB SAS12G SSD RAID10 (PCIe Gen3 x8) for Data

24 cores

96 GB

2 x 25 GbE

NiFi

  • Minimal

  • Proof of concept

  • Commodity

  • Balanced

  • Compute-intensive

Storage CPU RAM Network

400 GB SSD for OS

500 GB SSD for Data

8 cores

32 GB

1 x 10 GbE
Number of nodes Storage CPU RAM Network

2 or more

1 x 500 GB HDD for OS

1 x 500 GB SSD for Data

8 cores

32 GB

1 x 10 GbE
Number of nodes Storage CPU RAM Network

2 or more

2 x 500 GB SSD RAID1 for OS

4 x 500 GB SAS12G SSD RAID10 (PCIe Gen3 x4+) for Data

12 cores

64 GB

2 x 25 GbE
Number of nodes Storage CPU RAM Network

2 or more

2 x 500 GB SSD RAID1 for OS

8 x 500 GB SAS12G SSD RAID10 (PCIe Gen3 x8) for Data

24 cores

96 GB

2 x 25 GbE
Number of nodes Storage CPU RAM Network

4 or more

2 x 500 GB SSD RAID1 for OS

16 x 500 GB SAS12G SSD RAID10 (2 controllers PCIe Gen3 x8) for Data

48 cores

192 GB

2 x 25 GbE

ksqlDB

Load pattern Number of nodes Storage CPU RAM

All

2 or more

SSD (volume depends on the number of requests and aggregations)

4 cores

32 GB

REST Proxy

Load pattern Number of nodes Storage CPU RAM

All

2 or more

HDD (no speed requirement)

16 cores or more (work with producer/

consumer)

1 GB

64 MB per producer

16 MB per consumer

Arenadata Enterprise Tools

If you use ET along with your product, take into account the following requirements:

  • Virtual server.

  • CPU: 4 cores or more.

  • RAM: 8 GB or more.

  • Disk: 50 GB or more.

  • Disk space requirements:

    • Docker Root directory. Ensure there is sufficient free space in the directory where Docker stores images and temporary layers (/var/lib/docker by default). It is recommended to reserve at least twice the size of the uncompressed product packages, as Docker creates temporary layers during the loading process before the final image formation.

    • Installation directory. Ensure that the /opt/arenadata/etools/ directory (or another selected directory) has enough available disk space to store uploaded archives and service data.

Arenadata Monitoring

If you use ADM along with your product, take into account the following requirements:

  • Virtual server.

  • CPU: 8 cores or more.

  • RAM: 16 GB or more.

  • Disk: 10 GB per monitored host or more.

Found a mistake? Seleсt text and press Ctrl+Enter to report it