To split large files into smaller files in Unix, use the`split` command. At the Unix prompt, enter: split [options] filename prefix
Replace filename with the name of the large file you wish to split. Replace prefix with the name you wish to give the small output files. You can exclude -l linenumber -b bytes
0, or replace it with either of the following: -l linenumber -b bytes
If you use the -l linenumber -b bytes
1 (a lowercase L) option, replace -l linenumber -b bytes
2 with the number of lines you'd like in each of the smaller files (the default is 1,000). If you use the -l linenumber -b bytes
3 option, replace -l linenumber -b bytes
4 with the number of bytes you'd like in each of the smaller files.
The split command will give each output file it creates the name prefix with an extension tacked to the end that indicates its order. By default, the split command adds -l linenumber -b bytes
8 to the first output file, proceeding through the alphabet to -l linenumber -b bytes
9 for subsequent files. If you do not specify a prefix, most systems use split myfile
0.
Examples
- In this simple example, assume
split myfile
1 is 3,000 lines long:
split myfile
This will output three 1000-line files:
split myfile
2,
split myfile
3, and
split myfile
4. - Working on the same file, this next example is more complex:
split -l 500 myfile segment
This will output six 500-line files:
split myfile
5,
split myfile
6,
split myfile
7,
split myfile
8,
split myfile
9, and
split -l 500 myfile segment
0. - Finally, assume
split myfile
1 is a 160KB file:
split -b 40k myfile segment
This will output four 40KB files:
split myfile
5,
split myfile
6,
split myfile
7, and
split myfile
8.
For more information, consult the man page for the split command. At the Unix prompt, enter: man split
You may also wish to investigate the split -l 500 myfile segment
7 command, which splits files based on context. For more information, see the man page for the split -l 500 myfile segment
7 command. At the Unix prompt, enter:
Use a temporary directory, then find all your files, randomize the list with sort, and move the top 1000 of the list into the temporary directory. Delete the rest, then move the files back from the temporary directory. $ mkdir ../tmp-dir $ find . -type f | sort -R | head -1000 | xargs -I "I" mv I ../tmp-dir/ $ rm ./* $ mv ../tmp-dir/* .
If xargs complains about line length, use a smaller number with head and repeat the command as needed (ie, change -1000 to -500 and run it twice, or change to -200 and run it 5 times.)
It will also fail to handle filenames that include spaces; as @rld's answer shows, you can use find's -print0 argument, the find`0 arguments to `sort and head, and find`3 with `xargs to ensure proper filename handling.
Finally, if the `find`5 already exists, you should substitute a directory name that doesn't exist.
You probably know about using the up and down arrow keys to scroll through your Bash history, but did you know that there's a lot more to Bash history than just repeating commands? There is much more to the story. Or, should that be much more to the history? In either case, the history command is one of those obscure commands that is powerful and handy to know on at least a basic level. This article will take the mystery out of Bash history to make it a more friendly sysadmin tool.
The background
In Bash, your command history is stored in a file (.bash_history) in your home directory. The leading (.) makes the file hidden from normal view. To see it, issue the
$ echo $HISTSIZE 1000 $ echo $HISTFILESIZE 1000 $ echo $HISTFILE /home/khess/.bash_history
0 command.
$ ls -a . .. .bash_history .bash_logout .bash_profile .bashrc
You can operate on the .bash_history file as you would any other plain ASCII text file.
The three related environment variables you need to be aware of are HISTFILE, HISTFILESIZE, and HISTSIZE.
- HISTFILE—/home/<username>/.bash_history
- HISTFILESIZE—1000
- HISTSIZE—1000
The HISTFILE variable holds the name and location of your Bash history file. HISTFILESIZE is how many commands can be stored in the .bash_history file. HISTSIZE is the number of cached commands. Once you reach 1000 commands, the oldest commands will be discarded as new ones are saved.
$ echo $HISTSIZE 1000 $ echo $HISTFILESIZE 1000 $ echo $HISTFILE /home/khess/.bash_history
Each user, including root, has these variables and sizes assigned by default. If you want to change these variables, edit the .bashrc file in your home directory. For example, if you want to change the size variables to 500 commands, edit the
$ echo $HISTSIZE 1000 $ echo $HISTFILESIZE 1000 $ echo $HISTFILE /home/khess/.bash_history
2 file and use the following entries:
export HISTSIZE=500 export HISTFILESIZE=500
To use the new values without logging off and back on again, you can execute the
$ echo $HISTSIZE 1000 $ echo $HISTFILESIZE 1000 $ echo $HISTFILE /home/khess/.bash_history
2 file.
$ . ~/.bashrc
Your values are now active for your current shell and any subshells.
$ echo $HISTSIZE 500 $ echo $HISTFILESIZE 500
If you don't want your history to keep duplicated commands, you can instruct your history to ignore duplicate entries by adding the following to your .bashrc file:
export HISTCONTROL=ignoredups
The problem
The problem with Bash history is that it's not written to the .bash_history file until you log off. This makes it impossible to use the history command for scripting. For example, say you're teaching a Linux class, and you want to check to see if students have run a particular command to copy their files to a mounted external drive. If the student stays logged onto the system, you won't be able to determine if the copy has been performed because your script uses their history file (/home/student/.bash_history) to check if the command has run.
The solution
To work around this feature, use the write option for the
$ echo $HISTSIZE 1000 $ echo $HISTFILESIZE 1000 $ echo $HISTFILE /home/khess/.bash_history
5 command. As part of the class instruction, you'd have the students run this command to save their history even if they don't log off.
$ history -w
This writes all current session command history to the HISTFILE. And now your check script can easily find if the command has been run. You also have to trust that the student will run the history command.
[ Check out Seth Kenlon's history lesson: Make Bash history more useful with these tips ]
Another problem
Since the HISTSIZE and HISTFILESIZE are relatively large (1000 commands), it's possible that the student ran the command before. You can add the export HISTCONTROL=ignoredups entry into their .bashrc, but this won't help if the student ran a similar command two days ago because the file will only show a single command.
The solution to this problem
To work around this problem, you should clear the student's history upon logging on. Edit the .bashrc file again and add the following command to it.
history -c
This command clears their history on each log on and removes the problem of picking up a previously run command. You don't have to worry about picking up a command from previous sessions because the system logs off idle users after a specified period of time if you set the idle time limit in
$ echo $HISTSIZE 1000 $ echo $HISTFILESIZE 1000 $ echo $HISTFILE /home/khess/.bash_history
6. Edit the
$ echo $HISTSIZE 1000 $ echo $HISTFILESIZE 1000 $ echo $HISTFILE /home/khess/.bash_history
6 file and add the following entry.
TMOUT=300
The TMOUT variable uses a number of seconds of idle time. In this case, 300 seconds is five minutes. You can set this value to any number of seconds. This value will take effect on the next log on for all users. To set individual timeout values, use
$ echo $HISTSIZE 1000 $ echo $HISTFILESIZE 1000 $ echo $HISTFILE /home/khess/.bash_history
2 but realize that users may change this file at will.
[ Download now: A sysadmin's guide to Bash scripting. ]
Wrap up
Bash history is a handy tool, but it can be frustrating if you aren't familiar with its options and quirks. There are many more options available in history. Check the man pages for further information. And remember that each command you enter is history in the making.
How do I see top 100 lines in Linux?
Linux head command.
Using head to view the first 10 rows of text file. head myTextFile.txt..
Using head to view the first 100 lines. head -100 myTextFile.txt. You can use head and tail to view the exact lines of your file..
Using head to view lines 50-60 from a file that's 100 lines: tail -50 myTextFile.txt | head -10..
How do I print top 10 lines in Linux?
The head command is used to display the first lines of a file. By default, the head command will print only the first 10 lines. The head command ships with the coreutils package, which might be already installed on our machine. Mind that newlines, tabs, and spaces are also counted as bytes.
How to cat only 100 lines in Linux?
2 Answers.
head --lines=100 print the first 100 lines..
head --lines=-100 print all but the last 100 lines..
tail --lines=100 print the last 100 lines..
tail --lines=-100 print all but the first 100 lines..
How do you limit the number of lines of output in bash?
The -n option tells head to limit the number of lines of output. Alternatively, to limit the output by number of bytes, the -c option would be used.